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RAISING THE AGENCIES’ GRADES— PRO- 
TECTING THE ECONOMY, ASSURING REGU- 
LATORY QUALITY AND IMPROVING ASSESS- 
MENTS OF REGULATORY NEED 


TUESDAY, MARCH 29, 2011 

House of Representatives, 

Subcommittee on Courts, 

Commercial and Administrative Law, 

Committee on the Judiciary, 

Washington, DC. 

The Subcommittee met, pursuant to call, at 4 p.m., in Room 2141 
Rayburn House Office Building, the Honorable Howard Coble 
(Chairman of the Subcommittee) presiding. 

Present: Representatives Coble, Gowdy, Gallegly, Reed, Ross, 
Cohen, and Johnson. 

Also Present: Representative Conyers. 

Staff Present: (Majority) Daniel Flores, Subcommittee Chief 
Counsel; John Hilton, Counsel; Johnny Mautz, Counsel; Allison 
Rose, Professional Staff Member; Ashley Lewis, Clerk; (Minority) 
James Park, Subcommittee Chief Counsel; and Susan Jensen 
Lachmann, Counsel. 

Mr. Coble. Good afternoon ladies and gentlemen. The Sub- 
committee will come to order. 

As we strive for economic recovery, one thing is clear, overregula- 
tion and poor regulation can stunt economic growth; and, most im- 
portantly, job creation. Oftentimes when the Federal Government 
implements inefficient or unnecessary regulations, capital that 
could be used to invest in new jobs is alternatively used for compli- 
ance or withheld to cover anticipated regulatory costs. 

Recently, the Mercatus Center published the results of its regu- 
latory report card project which evaluated the government’s compli- 
ance with the rulemaking process and assessed agencies’ perform- 
ance formulating and promulgating regulations. The results regret- 
tably show that the Federal agencies are not doing an adequate job 
formulating and promulgating regulations. 

According to the Mercatus study, agencies routinely fail to imple- 
ment well, or even follow some of the basic steps of good rule- 
making practice, including practices prescribed by executive orders 
on regulation. As one can see from the detail and complexity of the 
Mercatus report, there is no silver bullet that will resolve all of the 
problems that have been created by ineffective or unnecessary reg- 
ulations. It is our hope, my hope, that we can extract a few com- 

( 1 ) 
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mon principles from today’s hearing that can he incorporated into 
future legislation that will improve regulatory consistency, effi- 
ciency, and predictability so that it will yield better regulation 
when it is needed. 

We will also explore today two potential reforms that already 
have begun to emerge from the results of the report card project, 
our earlier hearings, and even President Obama’s recent state- 
ments on rulemaking. The first reform would create an additional 
procedure in the rulemaking process before the agency has settled 
on its course of regulation. Professor Peter Strauss, a witness at 
our last hearing, told us that the agency commitments during this 
phase of rulemaking, before a proposed rule is even published, 
often convert the Administrative Procedures Act notice and com- 
ment procedures into nothing more than a farce. 

The second potential reform would implement stricter require- 
ments for agencies to demonstrate a need to regulate before it 
issues regulations. Common sense tells us that just because an 
agency can make a new regulation does not mean that it should 
make a new regulation. 

The first step in the process should be to ask whether a problem 
exists. If no problem requiring regulation does in fact exist, then 
the agency should proceed no further, it seems to me, to coin the 
old adage, “if it ain’t broke don’t fix it.” You have heard that many 
times. 

Congress must have assumed, when it enacted the APA, that 
agencies would only regulate when they could identify a problem 
that needed regulation. Executive orders, moreover, have long 
spelled out that agencies should identify specific market failures 
before they regulate. Astonishingly, however, the regulatory report 
card project showed that the single rulemaking step at which agen- 
cies performed the worst is demonstrating that there is a need for 
regulation at all. This suggests that it is time to include in the 
APA itself stricter requirements to demonstrate regulatory need. 
These and other reforms should help us to protect the economy and 
improve the quality of the regulatory agencies’ work. 

I look forward to hearing from our witnesses. I reserve the bal- 
ance of my time. 

I am pleased to recognize the distinguished gentleman from 
Michigan, the former Chairman of the Judiciary Committee, Mr. 
Conyers. 

Mr. Conyers. Thank you. Chairman Coble. We come together 
this afternoon for the fifth consideration of the subject of the bur- 
den of regulation on business. The title of the hearing is Raising 
the Agencies’ Grades — Protecting the Economy, Assuring Regu- 
latory Quality and Improving Assessments of Regulatory Need. 

This is a very weighty subject since in the interim, we have not 
been creating more jobs for Americans, unemployment is the last 
economic indicia to be affected positively as we try to move out of 
a recession and in some places, a depression in others. We are not 
dealing with the 4-year ongoing mortgage foreclosure crisis, giving 
agencies less resources to protect health and safety of the air we 
breathe and the food that we eat. 

And so I am beginning to wonder about the objective at the end 
and the effectiveness of a cost benefit analysis because these un- 
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verifiable assessments are probably as good an opinion as anybody 
else’s around, but it may very well not be dispositive. 

Now, we are in the process of trying to determine about the ef- 
fects of regulatory failure. You know, there were regulations in- 
volved in the Japanese meltdown. We just heard today that they 
discovered that there are leaks that are now increasing the fear of 
contamination since they have been found in the foodstuffs, and 
other environmental tragedies. 

Only last week we observed the 100th anniversary of that tragic 
New York fire that triggered so much regulation that we now are 
worried about overburdening businesses. And so the benefits of reg- 
ulation are not, to me, contemplated, and I invite my witnesses, 
our witnesses, to share this part of my presentation with the rest 
of the Committee because benefits frequently far exceed the costs 
of regulation. And so if we are only talking about costs in terms 
of dollars and cents, one can miss the full impact of regulation. 

I am hoping that this conversation will lead us to look at the in- 
credible number of activities in which tragedy occurred, since the 
Triangle Shirtwaist Factory fire which has come down through a 
lot of activities, going back to the exploding gas tanks in the Ford 
Pinto discovered by a young attorney, Ralph Nader, the Three Mile 
Island nuclear meltdown, major bus crashes where people died be- 
cause of a lack of regulating seat belts, coal mine explosions in 
West Virginia and so on. I will put the rest in the record. 

I welcome our witnesses to a genuine discussion about this mat- 
ter, and I thank Chairman Coble for the generosity and the time 
that has been allotted me. 

Mr. Coble. I thank the gentleman. 

[The prepared statement of Mr. Conyers follows:] 
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Statement of the Honorable John Conyers, Jr. 

For the Hearing on "Raising the Agency's Grades - Protecting the 
Economy, Assuring Regulatory Quality' and Improving Assessments 
of Regulatory Need" Before the Subcommittee on Courts, 
Commercial and Administrative Law 

Tuesday, March 29, 2011, at 4:00 p.m. 

2141 Rayburn House Office Building 


Today's hearing is now the fifth hearing that this 
Subcommittee has held on the subject of regulatory 
refonn. 


. As with the prior hearings, the subject of today’s 
hearing focuses on a solution in search of a problem 
based on faulty assumptions and conclusions. 

It would be so much more beneficial if the 
Majority focused its attention on serious problems, 
like creating more jobs for Americans, dealing with 
the now four-year long ongoing mortgage 
foreclosure crisis, and giving agencies more ^ not 
less - resources to protect the health and safety of 
the air we breath and the food we eat. 
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Instead, I get the strong sense that the Majority 
is following the same strategy it often does - repeat 
something often enough and people will begin think 
it is tme, whether it is or not. 

With respect to today’s hearing, that talking 
point seems to be that agencies have "overburdened" 
businesses with regulations that are stifling 
economic growth. 

My response remains the same. The Majority 
deliberately downplays the benefits of regulation 
and exaggerates its costs, when in fact, the benefits 
of regulation far exceed its costs, whether those 
benefits are defined in monetaiy terms or in terms of 
promoting values like protecting public health and 
safety and ensuring civil rights and human dignity. 


2 
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The particular focus of today's hearing is the 
so-called Regulatory Report Card issued by the 
Mercatus Center, an industry-funded think tank. 

These report cards purport to "grade" agencies' 
use of cost-benefit analysis. Perhaps not 
surprisingly, the Mercatus Center finds most 
agencies lacking in its admittedly subjective view. 

T am skeptical of basing major policy decisions 
solely on a subjective and unverified assessment like 
that. But this hearing gives us another opportunity 
to explore important questions about the costs and 
benefits of regulations. 

Agencies ought to retain the flexibility to head 
off problems and not wait for disaster to strike 
before regulating, and measures that needlessly 
hamstring that flexibility are dangerous. 


3 
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Last Friday was the 100th anniversary of the 
Triangle Shirtwaist Factory fire, in which 146 
workers - mostly immigrant women, some of them 
teenagers - were killed. That tragedy resulted 
directly trom the lack of any regulation regarding 
workplace safety, fire safety, and employee rights. 

Business owners at the time resisted attempts by 
government to reduce the risk of such a tragedy 
happening again in tenns eerily similar to what we 
hear from House Republicans today, claiming the 
new regulations were needless, useless, and would 
wipe out industry in the state. 


4 
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Since 1911, there have been at least 24 major 
examples of regulatory failure, including the sinking 
of the Titanic, the exploding gas tanks in the Ford 
Pinto, the Three Mile Island nuclear reactor 
meltdown, and the Union Carbide plant explosion in 
Bhopal, India. Just the most recent examples 
include: 

1 . The major bus crash in New York, where 1 5 
people were killed and a lack of seat belts may 
have contributed to the loss of life. 

2. The Massey coal mine explosion in West 
Virginia, which took the lives of 29 miners. In 
fact next month, will mark the one-year 
anniversary of that explosion. 


5 
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3. The explosion of BP's Deepwater Horizon oil rig 
in the Gulf of Mexico, which stenuiied from lax 
regulation of oil drilling platfonus, is only the 
most prominent example. 

4. The home foreclosure crisis, the 2008 financial 
crisis, and the ensuing Great Recession, all of 
which stemmed from the fact that regulators, 
under the Bush Administration, lacked the 
direction, resources and authority to confront the 
highly reckless behavior of the private sector, 
and particularly the lending and financial 
services industries. 

I raise these examples to make the point that we 
ought not wait until another fragedy on the scale of 
the Triangle factor)^ fire takes place. That fire was a 
dramatic illustration of w'hy government must 
sometimes regulate industry to protect people. 


6 
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I hope we keep that bigger picture in mind as we 
continue this ongoing discussion about the need for 
regulation. 

Another question we ought to discuss is whether 
the real problem with the regulatory system is that 
agencies lack the resources needed to fulfill their 
responsibilities that they have been tasked to 
perform by Congress. 

This possibility was brought home to me last 
month as I watched the Majority strip away funding 
for the Environmental Protection Agency and other 
agencies through a series of amendments to H.R. 1 , 
the "Full-Year Continuing Appropriations Act, 
2011 ." 



specifically, these amendments would prohibit 
these agencies from using any federal flmding to 
promulgate and implement various regulations, 
especially those concerning environmental 
protection, the implementation of the Patient 
Protection and Affordable Care Act, and the 
consumer protection provisions and other financial 
refomis of the Dodd-Frank Wall Street Reform Act. 

The Majority has forgotten that it was in direct 
response to these regulatory failures in the health 
care and financial realms that Congress passed the 
Dodd-Frank Act and other measures. 

Do we really want to set ourselves up again for 
the kind of “regulatory Wild West” tliat got us into 
trouble in the first place? 



12 


Finally, we should consider how to account for 
benefits that are difficult to reduce to dollar figures. 
For example: 

• What is the benefit of a rule that prohibits 
prison rape? 

• What is the benefit of a rule requiring 
wheelchair access to a public restroom? 

Reduced simply to dollar figures, the costs of 
such rules may indeed outweigh the benefits, given 
the relatively small number of people who would 
benefit from such rules. 

But the benefits of regulations may go well 
beyond what can be discussed in dollar terms. 


9 
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I, for one, believe that a rule that prevents even 
one person from being raped in prison or that allows 
one person to use a public restroom with dignity is. 
worth the monetary cost on private prison 
corporations and businesses that would constitute 
public accommodations under federal law. 

That is the fundamental role of govermiiental 
regulation and, indeed, of govermnent itself - to use 
its power to protect the public, especially those who 
are most vulnerable to the whims of coi*porations' 
profit-maximizing ethos. 

I thank our witnesses for being here today and 
look forward to their testimony. 
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Mr. Coble. We have now been joined by the distinguished Rank- 
ing Member, the gentleman from Tennessee, Mr. Cohen. 

Mr. Cohen. Thank you, Mr. Coble. Pardon my tardiness. 

Today we consider the regulatory report card project undertaken 
by the Mercatus — Mercatus Center, not exactly on the tip of my 
tongue on a regular basis. These report cards purport to assess the 
quality of agencies use of regulatory analysis by assigning numer- 
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ical grades from zero to five for each of 12 questions used to assess 
performance of agencies for a possible 60 points if you hit the top 
five for 12 times. The hearing title implies that Mercatus’ grades, 
“are accurate based on sound methodology.” 

The title also suggests we should, therefore, focus on changing 
the existing regulatory system based on this finding by the 
Mercatus Center. By Mercatus’ own admission, however, the report 
cards, “are subjective and its grading is not transparent or capable 
of any third-party replication.” 

Although Mercatus says it has instituted a process to address 
these concerns, that process appears to involve only Mercatus 
scholars verifying each others conclusions, not any objective third- 
party analysis and intervention. 

To the extent that the majority seeks to premise changes to the 
rulemaking process based only on Mercatus’ findings, I find this, 
and I think the Nation would find it troubling. Perhaps I would be 
more comforted if it were not for the fact that Mercatus does not 
approach the issue of regulatory reform with a neutral perspective, 
the way that maybe, say the Administrative Conference of the 
United States might approach something. Mercatus was founded 
and is funded by the Koch brothers, not the beverage that we all 
enjoy but Charles and David Koch, the owners of Koch Industries, 
the second largest privately held company in the country, a com- 
pany which has large oil and lumber interests among others. Oil 
and lumber are industries not normally desirous of any government 
regulation at all. They like to cut trees and decide when and how 
they will replenish their forests and take from the earth as much 
oil as they can, and we saw with Deepwater Horizon how good it 
is not to regulate oil drilling. Mercatus continues to be heavily 
funded by donations from some of the Nation’s largest corporations, 
all of which have an interest in stifling economic health and safety 
regulations. 

According to The Wall Street Journal, 14 of the 23 regulations 
that President Bush put on his regulatory hit list had been rec- 
ommended first by the Mercatus Center. A lawyer described 
Mercatus’ strategy this way: You take corporate money, you give it 
a neutral sounding think tank, hire people with pedigrees and aca- 
demic degrees who put out credible-seeming studies, but they all 
coincide perfectly with the economic interests of their founders, 
kind of like an academic middle person. Mercatus’ regulatory re- 
port card may or may not turn out to be accurate. The problem is 
we will never really know because there is no way to verify a sub- 
jective conclusion versus in-house doctoring. 

We need to guard against enacting what might turn out to be 
needless analytical requirements based on possibly faulty findings 
by a think tank with a known regulatory agenda and contributors 
who have a particular desired outcome that fiiey seek. 

As I have said before, agencies must retain the ability to act to 
protect Americans’ public health and safety and ensure the sound- 
ness of our Nation’s economy and to guarantee that Americans’ 
civil rights are not infringed upon. While recognizing that regula- 
tion can impose costs, we understand that, we should not ever for- 
get that the benefits far outweigh the costs. America has had some 
of its greatest years of economic and job growth under the current 
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regulatory system. At a minimum, that seems to point to the con- 
clusion there is no inconsistency in the regulatory system we have 
and economic and joh growth. We ought to keep that in mind and 
proceed cautiously before further hampering agency rulemaking. 

I yield back the balance of my time, and thank you for the allow- 
ance. 

Mr. Coble. I thank the gentleman from Tennessee. We have 
been joined by the distinguished gentleman from South Carolina, 
Mr. Gowdy, and the distinguished gentleman from Florida, Mr. 
Ross. Good to have you all with us. We will proceed with the hear- 
ing. I will give you some background on our witnesses who will ap- 
pear today. 

Mr. Richard Williams is the Mercatus Center Director of Policy 
Research. He served in the Office of Management and Budget for 
27 years as the director of Social Sciences and the Center For Food 
Safety and Applied Nutrition in the Food and Drug Administration. 
Dr. Williams is a expert in benefit cost analysis and risk analysis, 
particularly related to food safety and nutrition. He has published 
in risk analysis and the Journal of Policy Analysis and Manage- 
ment, and has counseled foreign governments, including the United 
Kingdom, South Korea, and Australia. A Vietnam veteran. Dr. Wil- 
liams received his Ph.D. and his MA in economics from Virginia 
Tech and his B.S. In business administration from the Old Domin- 
ion University. He has served as an adviser to the Harvard Center 
For Risk Analysis and taught economics at Washington and Lee 
University. 

Mr. Jerry Ellig is a senior research fellow at the Mercatus Cen- 
ter at George Mason University where he has worked since 1996. 
Between August 2001 and August 2003, he served as deputy direc- 
tor and acting director of the Office of Policy Planning at the Fed- 
eral Trade Commission. 

Dr. Ellig also has served as a senior economist for the Joint Eco- 
nomic Committee on the U.S. Congress and as an assistant pro- 
fessor of economics at George Mason University. Dr. Ellig directed 
the Mercatus Center’s regulatory report card project which assesses 
the quality of agency performance in promulgating major regula- 
tions. Dr. Ellig has published numerous articles on government 
regulation and business management in both scholarly and popular 
periodicals, and has coauthored and edited several books on com- 
petition, regulation, and environmental energy. He earned his 
Ph.D. degree and his M.A. in economics from George Mason Uni- 
versity and his B.A. in economics from Xavier University. 

Our third witness is Professor Robert L. Glicksman. Professor 
Glicksman has published widely on the subject of environmental 
and administrative law. Before coming to George Washington Uni- 
versity in 2009, he taught at the University of Kansas School of 
Law where he was the Robert W. Wagstaff distinguished professor 
of law. A graduate of the Cornell School of Law, prior to joining the 
academy. Professor Glicksman worked in private practice at a firm 
in Washington, DC where he focused on environmental, energy and 
administrative law issues. 

Professor Glicksman joined the Center Eor Progressive Reform in 
2002, and has sat on its board of directors since 2008. 
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Our three witnesses bring glowing credentials to the table. We 
are glad to have you all with us. We try to go by the 5-minute rule 
that we apply to you all, and we try to apply it to ourselves as well. 
You will see when the amber light appears, that is your notice that 
time is evading. You will have 1 minute after that. When the red 
light appears, if you could wrap up shortly thereafter, we would be 
appreciative. 

Dr. Ellig, if you would start us off. 

TESTIMONY OF JERRY ELLIG, SENIOR RESEARCH FELLOW, 
MERCATUS CENTER, GEORGE MASON UNIVERSITY 

Mr. Ellig. Thank you, Mr. Chairman, Ranking Member Cohen, 
Members of the Committee. My name is Jerry Ellig. I am a re- 
search fellow at the Mercatus Center at George Mason University. 
As the Chairman indicated in his introduction, I have also served 
in two out of three branches of the Federal Government. I probably 
won’t get into the third. But each time I have left government and 
gone back to academia, I have walked out with a long list of stud- 
ies I wished someone had done, experts I wished we had been able 
to consult if we had only been able to find them, to answer ques- 
tions in order to make better policies and make better decisions in 
government. And that is really the genesis of the Mercatus regu- 
latory report card, trying to figure out what is it that agencies ac- 
tually do when they sit down to make decisions about regulation, 
and how well do agencies do the things that Presidents of both po- 
litical parties have been telling them to do for several decades. 

Some time ago in our schooling, most of us probably learned that 
there are a few basic things that we are supposed to do before 
making important decisions that affect us or affect the lives of 
other people. Really basic things, like identify the goal that we are 
trying to achieve, what outcome do we want, identify the nature of 
the problem we have to overcome to achieve the goal. Identify the 
various alternative ways of achieving that goal, and then weigh the 
pros and cons of alternatives. You might call that Decisionmaking 
101 . 

Well, regulatory analysis, as required by Federal executive order, 
is simply Decisionmaking 101 applied to regulation. What we are 
trying to do in the Mercatus regulatory report card is assess how 
well agencies do these basic things that you would do before mak- 
ing any big decisions. We have examined all of the proposed eco- 
nomically significant regulations issued over the past few years, 
those are the really big ones. We used criteria drawn from the ex- 
ecutive order that governs regulation, an 0MB Circular 8-4 that 
lays out best practices for regulatory analysis. We look at the qual- 
ity of the analysis, and we also look at the extent to which the 
agency claims to have used the analysis when it made decisions 
about the regulation. 

So what do we find? We find that agencies do a lot of good things 
in their regulatory analysis. We also find that the average quality 
is low, the best ones are not stellar, there is wide variation in the 
quality of regulatory analysis, we see a lot of best practices in 
agency regulatory analysis, but they are not widely shared and no 
analysis does everything well. And we also see that often the regu- 
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latory analysis produced by agencies reads as if it were written 
after the major decisions about the regulation were made. 

You might call this the ready fire aim approach to regulation. 
And these findings are consistent with the findings of other schol- 
ars at other institutions, other universities, resources for the fu- 
ture, other respected places, who have looked at smaller groups of 
regulations to try to figure out what is the quality of the analysis 
and what do agencies do with it. 

Most importantly for the topic of this hearing, the biggest single 
deficiency we find in many agency regulatory analyses, not all, is 
insufficient definition and an explanation of the systemic problem 
that the regulation is supposed to solve. Now, that is a big mouth- 
ful of jargon. Let me give an analogy. 

A couple of years ago I walked into the bathroom and found 
water on the floor. That wasn’t the problem, that was the symp- 
tom. We had to do some analysis to solve the problem. We found 
out that there was a crack in a plastic pipe that, in turn, was 
caused by the fact that the toilet wasn’t leveled and it was rocking 
back and forth and that is what cracked the pipe. After we did the 
analysis, we could solve the problem at minimal cost. 

Now when I sit down to read agency regulatory analyses, they 
frequently read like somebody walking into a bathroom saying 
well, the problem is obvious, there is water on the floor. And the 
solution is obvious. We are going to make everybody buy a mop, 
and we will now take public comment on what types of mops we 
should require people to buy and how long the handle should be. 
Anyone who disagrees with the favored approach is accused of 
wanting to allow children to slip on wet floors. 

Now, lest you think I am exaggerating, I have examples in my 
written testimony of a number of cases where we read agency regu- 
latory analyses looking for the definition of the systemic problem; 
and essentially, there is either an assertion of a problem with no 
underlying cause-and-effect theory, no underlying empirical anal- 
ysis, a symptom gets misdiagnosed as a problem, or the problem 
is simply stated as the purpose of this regulation is to implement 
such and such Public Law. 

More broadly, about half the regulations we looked at scored a 
zero or a one on this criterion, indicating that there was a little bit 
of a perfunctory look at a problem or an assertion, but not much 
real analysis. Now, some did well; but about half of them just 
didn’t do much. 

We also find when we looked at the quality of the analysis that 
there isn’t much difference across Administrations. So this is not 
a partisan issue or a political problem, it is an institutional prob- 
lem that can only be solved with changes in the incentives that 
agencies face to do good analysis. So instead of ready-fire-aim, the 
system should be look before you leap. 

Thank you for your time. 

Mr. Coble. Thank you. Professor. 

[The prepared statement of Mr. Ellig follows:] 
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Mr Chairmau aitd members of the subcommittee: 

'Hiank you for the oppommilv to present (his tcstimon\ I am an economist and research fellow at (he 
Mcrcatus Center, a 50Uc)(3) research, educational, and outreach oi^anization affiliated \Mth George 
Mason University in Arlington. Viigmin * M\ principal research for die lost 25 vears has focused on the 
regulatory process, govcmmcni pcrfomiance. and the effects of government regulation For this mason. 
I'm delighted to testily on today's topic. 

Somewhere along the line in our schooling, most of us Icam about a fov basic steps to lake before making 
a major decision These steps include ( 1 ) define the goal or goals v ou am try mg to achieve. (2) 
understand the nature of the obstacles to be overcome or problems to be solved to achieve the goal. (3) 
develop a list ofaltcmative ways to solve the problem, and (4) assess the pros and cons of each 
alternative After taking these steps, deciding on a course of action may still be challenging. The decision 
may involve difficult tradeoffs between some of the pros and cons But at least the decision will be 
informed by know ledge of the likely consequences of alternative actions Call diis "Decision Making 
101 '* 1 don't know any reasonable person who would argue we should m/f do these (hmgs when making 
decisions (hat have really big effects on our lives or the lives of others 

Regulatory impact analysts is nothing more than Decision Making 101 applied to regulation ( I ) define 
die outcome or outcomes the regulatory agency seeks to achieve. <2) understand the root causes of the 
problem that stands in the way of achieving the desired outcomes. (3) develop a wide variety of 
alternative ways to solve the problem, and (4) assess the pros and cons ofcach altcmabvc If we cut 
through all the jargon and details, these are the four main elements of regulatory analysts 

We expect federal ragulation to accomplish a lot of important dungs, such as protecting us from financial 
fraudsters, preventing woikpiace injuncs. presen mg clean air. and detemng terrorist anacks And 
regulation also requims sacrifices Depending on the regulation, consumers may pay more, workers may 
mceivc less, our rcliremcnl savings may grow more slowly due to mduced corporate profits, and vve may 
have less personal freedom Regulatory analv sis is the key ingmdient that makes these tradeoffs more 
transparent to decision makers So. understanding the effects of regulation has to start with sound 
regulatory analysis 


* This testimony reflects only ihe views of its author and does not represent an official posnlon of George Mason 
University 
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For more than three decades, executive orders have instructed federal agencies to conduct regulator}- 
impact analyses and consider the results of those analyses when making decisions. On Januaiy 1 8, 
President Obama issued Executive Order 13563, "Improving Regulation and Regulatory Review.’'^ 
Executive Order 1 3563 "reaffirms the principles, structures, and definitions governing contemporaiy 
regulatory review that were established in Executive Order 12866." In fact Executive Order 12866 
reaffirmed the principles originally established 30 years ago with Executive Order 12991 . 

This reaffinnation is welcome. The analytical principles in Executive Order 12866 and the Office of 
Management and Budget's accompanying guidance in Circular A-4' arc sound. The administration's 
reaffirmation of those principles may' help quell some uncertainty about future standards for regulatory- 
review that has existed since the administration announced in January 2009 that it planned to revise the 
executive order."^ 

But how well do executive branch agencies do what presidents have been telling them to do for more than 
three decades? Scholarly research on regulatory analysis, including the Mercatus Center's own 
Regulatory- Report Card, finds that agency regulatory^ analysis is often incomplete and seldom used in 
decisions. This pattern persists across administrations, suggesting that the source of the problem is 
institutional, not political. Fundamental institutional reforms arc necessary-- to ensure that agencies conduct 
high-quality regulatory- impact analy-sis and use it in decisions. In short, regulatory- impact analysis needs 
to be (1) required. (2) objective, and (3) used. 

Tlic body of my testimony documents current problems with the quality and use of regulatory- analy sis 
and suggests some solutions. Let me briefly summarize my recommendations: 

1 . Regulatory impact analysis should be required. Regulatory' analysis needs to be legislatively required 
for all federal agencies, including independent agencies. 

2 . Regulatosy impact analysis should be objective. All too often, regulatory- analyses read as if the agency 
first made most of the major decisions about the regulation, then handed the regulation off to its 
economists to produce an analysis to get tlie regulation tlirough tlie 0MB review process. Agencies 
should publish regulatory- analysis, along with all undcrly-ing data and research, before writing proposed 
regulations. Agency economists should have tlie independence to conduct objective analysis, instead of 
being expected to produce an advocacy document thatjustifics decisions that have already been made. 

3. Regulatory Impact analysis should be used. When Congress requires regulatory- agencies to consider 
particular factors in designing regulations, such as costs or efficiency, agencies usually' explain how those 
factors affected their decisions. Congress should require all agencies to explain, when proposing 
regulations, how- the major elements of regulatory- analysis affected decisions about the regulation. 

The Problem: Decisions Made Before Analysis 

Presidential executive orders on regulatory- review have had a limited effect on the quality- and use of 
regulatory- analy-sis. Case studies document instances in which regulatory analysis helped improve 
regulatory- decisions by providing additional options regulators could consider or unearthing new 


^ Executive Order 13563, “Improving Regulation and Regulatory Review,” Federal Register 76:14 (Jan. 21, 201 1 ), 
3821-23, http://www.whitehouse.gov/sites/default/files/omb/inforeg/eo12866/eo13563_01 18201 1.pdf. 

^ Office of Management and Budget, Circular A-4, “Regulatory Analysis” (Sept. 17, 2003), 
http://www.whitehouse.gov/sites/default/files/omb/assets/regulatory_matters_pdf/a-4.pdf. 

Office of Management and Budget, “Federal Regulatory Review: Request for Comments," Federal Register 74:37 
(Feb. 26, 2009), 8819, 

http://www.reginfo.gov/public/jsp/EO/fedRegReview/OMB_FR_Notice_on_Regulatory_Review.pdf. 
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infonnation about benefits or costs of particular modifications to tlie regulation.' But Government 
Accountabilit\ Office studies and scholarly research reveal that in many cases, rcgulatoiy impact analyses 
are not sufficiently complete to ser\-^e as a guide to agenc}’ decisions. The qualit}’ of anah'sis varies 
widely, and even the most elaborate analyses still have problems.'^ 

All too often, agency economists have to conduct regulatoiy- analysis after most major decisions about 
regulations have already been made. The analysis then becomes an advocacy document written to justify 
tlie agency's decisions, or a mere paperwork exercise to fulfill requirements imposed by tlie Office of 
Management and Budget. Surv^cying die scholarly' evidence on regulatory analy sis, Robert Hahn and Paul 
Tetlock conclude that economic analy'sis has not had much impact, and tlie general quality of regulatory- 
analysis is low. "Nonetheless,"’ they note, "in a world where regulatory impacts arc frequently measured 
in the billions of dollars, margins matter. Tlius, economists should pay more attention to how economic 
analysis can contribute to improving benefits and costs on the margin.”^ 

Most previous research examines subsets of economically significant regulations — often health, safety , 
and environmental regulations. Since 2008. the Mercatus Center’s Regulatory Report Card has assessed 
the quality and use of regulatory analy sis for all proposed, economically significant regulations issued by 
executive-branch agencies.^ 

We assess how well die agency defines and measures the outcome the rcgidation is supposed to produce, 
identifies and assesses tlie root cause of a market failure or otlier systemic problem tlie regulation seeks to 
solve, develops altcmativc approaches, and identifies the costs and benefits of the regulation. We evaluate 
die transparency, clarity', and documentation of models and data in the analysis. Finally, we assess the 
extent to which the agency used the analysis to make decisions and made provisions for retrospective 
analy-'sis of the regulation. In short, we examine how well the executive-branch regulatory-' agencies do 
what presidents have been telling them to do for more than three decades. 

The attached paper 1 coauthored with John Morrall, a 29-y ear veteran of the Office of Information and 
Regulatory-' Affairs who joined Mercatus as an affiliated senior scholar in 2010. summarizes the 


^ Winston Harrington, Lisa Heinzerling, and Richard D. Morgenstern (eds.), Reforming Regulatory Impact Analysis 
(Washington, DC: Resources for the Future, 2009); Richard D. Morgenstern, Economic Analyses at ERA: 
Assessing Regulatory Impact (Washington, DC: Resources for the Future, 1997); Thomas O. McGarity, 

Reinventing Rationality: The Role of Regulatory Analysis in the Federal Bureaucracy {Uev<j York: Cambridge 
University Press, 1991). 

® See Art Fraas and Randall Lutter, “The Challenge of Improving the Economic Analysis of Pending Regulations: 

The Experience of OMB Circular A-4” (Resources for the Future Discussion Paper 10-54, December 2010); Jamie 
Belcore and Jerry Ellig, “Homeland Security and Regulatory Analysis: Are We Safe Yet’’," Rutgers Law Journal 
(Fall 2009): 1-96; Robert W. Hahn, Jason Burnett, Yee-Ho I. Chan, Elizabeth Mader, and Petrea Moyle, 

“Assessing Regulatory Impact Analyses: The Failure of Agencies to Comply with Executive Order 12,866," Harvard 
Journal of Law and Public Policy 23. no. 3 (2001): 859-71 ; Robert W. Hahn, and Patrick Dudley, “How Well Does 
the Government Do Cost-Benefit Analysis?" Review of Environmental Economics and Policy 1, no. 2 (2007): 192- 
211; Robert W. Hahn, and Robert Litan, “Counting Regulatory Benefits and Costs: Lessons for the U.S. and 
Europe,” Journal of International Economic Law 8, no. 2 (2005): 473-508; Robert W. Hahn, Randall W. Lutter, and 
W. Kip VIscusl. Do Federal Regulations Reduce Mortality? (Washington, DC: AEI-Brookings Joint Center for 
Regulatory Studies, 2000); Government Accountability Office, Regulatory Reform: Agencies Could Improve 
Development, Documentation, and Clarity of Regulatory Economic Analyses, Report GAO/RCED-98-142 (May 
1 998); Government Accountability Office, Air Pollution: Information Contained in EPA’s Regulatory Impact 
Analyses Can Be Made Clearer, Report GAO/RCED 97-38 (April 1997). 

^ Robert W. Hahn and Paul C. Tetlock, “Has Economic Analysis Improved Regulatory Decisions?” Journal of 
Economic Perspectives 22, no. 1 (Winter 2008): 67-84. 

® Regulatory Report Card, Mercatus Center at George Mason University, www.mercatus.org/reportcard. 
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Regulator}- Report Card results for 2008 and 2009.'^ We assign a score to economically significant rules 
ranging from 0 to 5 points on 1 2 different criteria, for a total possible score of 60 points. In both 2008 and 
2009, agency regulator}-’ analyses earned an average of about 27 out of a possible 60 points, or 45 percent. 
If these were student papers, the average would be an ”F.'’ 

The highest score in 2008 was 43 points (72 percent), w'hich the Department of Transportation earned for 
its proposed Corporate Average Fuel Economy regulation. The highest score in 2009 was 48 points (80 
percent), for the combined DOT-EPA Corporate Average Fuel Economv and Greenliouse Gas Emission 
standards. The lowest score in 2008 w-as 7 points, forthc Social Security Administration’s regulation on 
scheduling administrative law judges. The lowest score in 2009 was 3 points, for a Department of Energy 
regulation on loan guarantees. Tlicsc latter two regulations arc both budget regulations that affect how- 
federal agencies implement spending programs. Budget-related regulations tend to receive much less 
thorough analysis and usually receive lower Report Card scores than most other kinds of regulations. 
These findings are all consistent with previous studies b\' academics and the Government Accountability 
Office that assess how- well regulatory agencies comply with the executive orders governing regulator}-’ 
analysis. 

Table 1 shows average scores on each of our 12 criteria in 2008 and 2009. In general, the analyses score 
tlie best on tlie entena tliat are easiest to satisfy, such as accessibility via tlie hitemet, documentation of 
data and models, and clarity (criteria 1-4). 


Table 1: Average Quality of Regulatory Analysis is Low 


Criterion 

2008 

Average Score 

2009 

Average Score 

Openness 



1 . Accessibility 

3.53 

4.06 

2. Data documentation 

2.24 

2.50 

3. Model documentation 

2.33 

2.62 

4. Clarity 

2.93 

2.83 

Analysis 



5. Outcome definition 

2.36 

2.38 

6. Systemic problem 

1.80 

1..6Q 

7, Alternatives 

2.29 

2.21 

8, Benefit-cost analysis 

2,09 

2.19 

Use 



9. Some use of analysis 

2.44 

2.24 

10. Considered net benefits 

2.20 

1.62 

1 1 . Measures and goals 

1.36 

1.29 

12. Retrospective data 

1.73 

1.50 




Total 

27.31 

27.02 


Maximum possible score on each criterion is 5 points. 
Maximum possible total score Is 60 points. 


® Jerry Ellig and John Morrall, “Assessing the Quality of Regulatory Analysis: A New Evaluation and Data Set for 
Policy Research” (working paper, Mercatus Center at George Mason University, Dec. 2010), 
http://mercatus.org/publication/assessing-quality-regulatory-analysis. 
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Quality of Analysis 

Few of the scores on individual criteria changed much between 2008 and 2009. There is some evidence 
that scores improved on some of the Openness criteria, such as accessibility and documentation — 
consistent with the Obama administration’s focus on transparency in the regulator}’ process. On average, 
explanations of how regulator}' costs affect prices of goods and seiv'ices also improved somewhat. Veiy 
modest improvements occurred in evidence of regulatory benefits and analysis of the distribution of 
benefits, hi general, tliese changes involved improvements from poor scores to middling scores. This is 
why the score on criterion 8, benefit-cost analysis, increased slightly. 

One of the major areas where regulator} analysis is weakest is identification of the systemic problem the 
regulation is supposed to solve (criterion 6). This is a key weakness. A systemic problem is a widespread 
problem that can be traced to a defect in the "rules of the game” that govern behavior — as opposed to the 
faults of a few "bad actors” that can be dealt with on a case-by-case basis. If the agency cannot identify 
and demonstrate the existence of a systemic problem that a regulation might solve, how can it assess 
whether the regulation is likel}^ to solve the problem or identif}- alternative solutions that might be more 
effective? Given the low score on this criterion, it is perhaps not surprising that average scores are also 
relatively low on other criteria that assess the "meat and potatoes” of the analysis — definition of the 
outcome the regulation is supposed to accomplish (criterion ?), identification and assessment of 
alternatives (criterion 7), and assessment of costs and comparison of costs with benefits (criterion 8). 

A homely analogy illustrates why thorough analysis of the systemic problem is important if regulatory 
agencies are going to do the job Congress expects tliem to do. A few years ago at home, I found water on 
the floor of the bathroom. Over a few days, we had to employ a little trial-and-crror scientific method to 
figure out whether the w^ater came from a leak in the sink, a leak in the toilet, a leale in a valve supplying 
water to the sink or the toilet, or a leak in a pipe. We finally found a crack in the plastic cold water pipe. 
We also determined that the emek itself occurred because the toilet was not completely level; it rocked 
back and forth a bit, putting pressure on the pipe. Anned with this knowledge of the systemic problem, 
w c replaced the broken piece of pipe and leveled the toilet. 

All too often, agencies go no further in analyzing the systemic problem than saying, “Look, there’s water 
on tlie floor.” Since tiiis definition of the problem is considered obvious, there is no perceived need to 
have a theory of how the water got there or evidence that the theory is true, and only solution considered 
is, "Buy a mop” — an expenditure that may be umiecessary’ (since an old rag will work just as well) and 
doesn't really get to the root cause of the problem. To add insult to injury, anyone who wants to do more 
careful analysis gets accused of wanting to let children slip on wet floors. 

Let me provide a few examples from specific regulations that illustrate the best and worst practices we’ve 
seen in analysis of the systemic problem. 

Best Practice: HUD Proposed RESPA Regulation. The maximum possible score on entenon 6, definition 
of the systemic problem, is 5 points. Tlius far, tlic only regulation we’ve evaluated that received 5 points 
on tliis criterion was a regulation proposed by the Department of Housing and Urban Development in 
2008 under the Real Estate Settlement Procedures Act. Tlic regulation would have revised the way certain 
real estate settlement charges related to mortgages are disclosed to consumers. The intended result was to 
reduce settlement costs for some consumers by making the charges easier to understand and compare 
across different lenders.^" 

In defining the problem, HUD's regulatory analysis suggested that the complexity^ of real estate 
transactions and lack of infomiation by some borrowers allow mortgage providers to collect higher fees 
from less informed or less sophisticated borrowers. Charging different customers different prices is not 
necessarily evidence of a “market failure,” because it does not necessarily lead to economic inefficiency. 
Car dealers, universities, and airlines often charge different customers different prices based on the 


A full set of evaluation notes on this regulation Is available at http://mercatus.org/reportcards/real-estate- 
settlement-procedures-act-proposed-rule-slmpllfy-and-lmprove-process. 
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customer's sophistication, knowledge, or perceived willingness to pay: tliis practice allows the companies 
to cover their fixed costs without chasing away the price-sensitive customers. But the practice strikes 
many people as unfair when the seller charges some customers higher prices simply because they are less 
informed. It is arguably inefficient if the transaction or the disclosures are so complex that a significant 
subset of customers does not understand them well enough to compare competing loan offers. 

Whether the problem is inefficiency or inequity or both, HUD's analysis identified a systemic root cause: 
asymmetnes in information tliat are exacerbated by the current way certain loan terms are disclosed. The 
analysis offered a coherent theory^ explaining how the infomiation problem could allow mortgage 
providers to charge some customers higher fees tlian others. It even explained why this pricing practice 
might not produce a "smoking gun“ of excessive profits for mortgage lenders or brokers: the finns may 
find they have to pay out most of the rewards to salespeople who are especial!}' skilled at inducing less- 
informed customers to over-pay for loans. In addition to a coherent theory , HUD offered empirical 
evidence that the theoiy-’ is actually true. The analysis cited several studies by government entities and 
consulting finns that found consumers with less education, no financial counseling, or more complex 
shopping strategies tended to pay more for loans and settlement services. About the only faults w'e could 
find with HUD's analysis of the systemic problem were that one study with results contradictory to 
HUD's was merely mentioned in a footnote rather than fully addressed, and the analysis did not 
completely assess uncertainties about tlie existence or size of tlie problem, Nevertlieless, HUD's treatment 
of the systemic problem is the best we hav'c seen thus far. 

Poor Practice: E-Verify in Federal Acquisition Regulations. This 2008 regulation required federal 
contractors to use die E-Verify system to ensure that they do not hire illegal workers. Tlie closest the 
analysis came to identifying a systemic problem was asserting that federal contractors hire illegal workers 
because they do not "internalize" all of the costs associated with having a less stable labor force. This 
assertion is supported with neither a coherent theorv’ nor empirical evidence.” 

Poor Practice: Side-Window Air Bag Standards. In 2009, the National Highway Traffic Safety 
Administration proposed standards for side window air bags to prevent passengers from being ejected in 
accidents. Tlie accompaiiving analysis offered no explanation of why auto manufacturers w'ould decline to 
include a seemingly eost-cffective safety measure, why consumers would be unwilling to pay for safety-, 
or why- effects on third parties make the regulation desirable even if manufacturers and consumers are 
unwilling to pay for air bags that meet the new standards. In fact, the analy sis even documents many 
tilings manufacturers are already doin^ to protect passengers from side-w-indow ejection, including 
installation of side-window air bags.” 

Poor Practice: Electronic Health Record Incentive Payments. In 2010, the Department of Health and 
Human Services proposed a rule implementing incentive payments to Medicare and Medicaid providers 
who adopt certified clcctronie medieal rceord teclmology. Hie aeconipanying analysis offers no definition 
or exammation of whatever sy-stemic problem prevents health care providers from adopting this 
technology on their own, Hiere are a few hints that the technology might have benefits to society that 
may not be captured by providers or patients, but tliis is not elaborated into a tlieory-, and no evidence is 
presented to support such a theory,” In the section labeled "Need for the Regulation," HHS simply states, 
‘This proposed rule would implement the provisions of the Amcriean Recovery and Reinvestment Aet of 
2009 (ARRA) (Pub. L. i 1 1-5) that provdde incentive payments to eligible professionals (EPs) and 


A full set of evaluation notes on this regulation is available at http://mercatus org/reportcards/employment- 
ellgibility-verification. 

A full set of evaluation notes on this regulation is available at http://mercatus.org/reportcards/motor-vehicle-safety- 
standards-ejection-mitigation. 

A full set of evaluation notes on this regulation is available at http://mercatus.org/reportcards/electronic-health- 
record-incentive-program. 
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eligible hospitals participating in Medicare and Medicaid programs that adopt and meaningfully use 
certified electronic health record (EHR) tcclinology.”''^ 

Tn numerous cases, agencies do not even offer particularly strong assertions about a problem the 
regulation is supposed to solve. Tliis happens most frequently witli regulations implementing federal 
spending programs, like the electronic health records regulation. Tlie regulatory- analysis simply caleulates 
the expenditures, and perhaps some benefits, with no analysis of the problem the expenditures are 
supposed to solve. Non-budget regulations also suffer from this deficiency on occasion, For example, a 
pair of regulations the Department of Justice proposed in 2008 to implement the Americans with 
Disabilities Act simply stated that the purpose of the regulations was to implement the act, with no 
explanation of why facility owners have made choices that sometimes conflict with the standards in the 
proposed regulations. 

The regulations Tve just mentioned may in fact solve some kind of systemic problem, but reading tlie 
agencies' regulatory analysis, T can’t tell what it is. When 1 put more effort into understanding a leak in 
my bathroom than some regulator}' agencies put into understanding the root causes of the problems 
they're supposed to solve with really big regulations, there's something serious!}' wrong with our 
regulatory-’ process. 

Use of Analysis 

The Regulatory Report Card research team also searches the Federal Register notice for the proposed 
regulation to see if there is any evidence that the agency used information about the systemic problem, 
projected regulatory outcomes, alternatives, benefits, or costs to malcc decisions. We do not expect the 
analysis to dictate tlie decision via a rigid rule, such as "regulate only w'hen monetized benefits exceed 
monetized costs." Section I of Executive Order 12866 explicitly instructs agencies to regulate only when 
the benefits yjustif}'" the costs, unless tlie law requires another approach. Tlius, the results of the 
regulator}' anal} sis are supposed to inform the decision, not determine the decision. Indeed, information 
about projected benefits or regulator}' alternatives could affect the agency’ s decision even if the agency is 
prohibited from considering costs (as with rules implementing the Clean Air Act). In these kinds of 
situations, we give agencies credit for using parts of the analysis, even if the agency did not use, or was 
prohibited from using, information about costs. 

We look to see whether any of tlie information prepared for the regulatory analysis appears to have had 
any effect on the agency’s decisions. This approach might generate some "false positives’' by giving 
agencies credit for using the analysis even when decisions ivere made for other reasons. Either way, 
agencies should be transparent about whether and how they have used regulator}-' analysis, and our project 
is the first to s}'stematically try and determine this fact. We liave found examples where agencies 
explicitly credit the regulatory analysis for affecting some significant decisions. But the average scores on 
our Use criteria are relatively low — less than 2 .5 out of a possible 5 points on each of these criteria. Even 
under our relative!} liberal definition of "use," agencies claim to use the regulatory impact analysis for 
significant decisions only about 20 percent of the time at best; 

• In 2008, agencies claimed tlie analysis affected a major decision for only 10 out of 45 proposed 
regulations. 

• In 2009, analysis affected a major decision for only 9 out of 42 proposed regulations. 

• In 2008, agencies chose the alternative that maximized net benefits or explicitly explained why 
the} chose another option for 1 1 regulations. 

• In 2009, they did so for 6 regulations. 


Department of Health and Human Services, “Medicare and Medicaid Programs; Electronic Health Record 
Incentive Program; Proposed Rule," Federal Reg/sfer75:8 (Jan. 13, 2010), 1975. 
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Scorus an: even loner on entena 1) and 12.nhicli indicate nhcdicr the agenev commits to using dtc 
results of ivgulator> onalvsis in (he future by establishing goals and measures for the regulation's 
outcomes and traclung data to measure tlic regulation's pcrfbnnonce. Tlie avemge secies on these cnierta 
are barely above I This indicates that on average, agency regulatory :uialyscs provide some semblance 
of a fnuncvvoi1( for evaluating the legulotion's effects on the future, but agencies have made no 
commitment to do so. 

In a few eases, agencies do articulate goals for ivgulations. CNpiain how legulaUons arv linked to agency 
strategic goals under the Gov enunent Performance and Results Act. commit to gathering data for 
a'trospcctivc c\ aluation. or commit to some form of rctrospccuvc review in the future Usually . however, 
they say linlc about this In 200H. we (bund only 4 prDpo.scd economically significanl regulations for 
which the agency established goals or measures for any major outcome the regulation was supposed to 
produce In only two cases did the agency enumerate (he data it would use to evaluate major outcomes the 
regulation was supposed to produce For 16 out of 4.^ regulations, (he agencies indicated that they might 
conduct some type of analyses in tlie future to assess some of the regulation’s effects. Similarly, in 2t)()9, 
we found 4 proposed regulations for which the agency established goals or measures fora major outcome 
and 2 regulations Ibr w hich the agency indicated vs hat data it would use to evaluate major oulcome.s of the 
regulation in the luturc Agencies indicated they might conduct some kind of retrospective analy sis for 
onlv 14 out of 42 proposed regulations 

The use of analysis in regulatory dcci.sions decreased m 20(i4 This result is apparent from (lie average 
scores on criteria 9- 12 for 2008 and 2(K)9 It becomes even stronger when vve pcrfomi cconometnc 
anal) sis that controls for (he quality of regulatory analy sis and the shiR in the miN of regulations proposed 
in (he two years. 

However, it would be a mistake to portray the first year of the Obama administration as a retreat from 
stellar use of analysis in die Bush administration Figure I shows die distnbution of Use scores in 200K 
and 2009. Neither year shows more tlion middling use of analysis. The principal difterciicc ts lliat the 
middle calcgoiy slinnks in 2(Xi9. with more proposed regulations that cither Ibil to use the analysis or 
make only a passing reference to it If agencies do not spectficallv let the public know how or whether 
they have used these analyses, the public is led largely in the dad about how their decision were made. 


Figure 1: Distribution of Scores for Use of Regulatory Analysis, 2008 and 2009 
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The good news is that foreven^ criterion, a few regulaton' analyses received a score of “5” for employing 
potential best practices, as table 2 shows. Tlic line of table 2 labeled '"Tlicorctical Highest Score” shows 
the score that could have been achieved in each year if one analysis had incorporated all of the best 
practices. Clearly, there is potential for tremendous improvement in the qualitv' of regulatoiy-' analysis 
simply through better dissemination of best practices across agencies. 

The knowledge required to produce better regulatory' analysis exists, dispersed throughout agencies in the 
federal government. 0MB Circular A-4 also summarizes a great deal of this knowledge. What's lacking 
arc institutional incentives to produce good analysis and use it to guide decisions. 


Table 2: Diffusion of Best Practices Could Greatly Improve Average Quality 


Criterion 

2008 

Average 

Score 

2008 

Highest 

Score 

2008# 

Earning 

Highest 

Score 

2009 

Average 

Score 

2009 

Highest 

Score 

2009 # 
Earning 
Highest 
Score 

Openness 







1. Accessibility 

3.53 

5 

12 

4.06 

5 

14 

2. Data documentation 

2.24 

5 

1 

2.50 

5 

5 

3. Model documentation 

2.33 

5 

3 

2.62 

5 

1 

4. Clarity 

2.93 

5 

3 

2.83 

4 

10 

Analysis 







5- Outcome dofinition 

2.3B 

5 

2 

2.38 

5 

1 

6 Systemic problem 

1.80 

5 

1 

160 

4 

4 

7. Alternatives 

2,29 

-5 

1 

2.21 

s 

1 

8. Benefit-cost analysis 

2.09 

4 

3 

2.19 

5 

1 

Use 







9. Some use of analysis 

2.44 

5 

2 

2.24 

5 

1 

10. Considered net benefits 

2.20 

5 

2 

1.62 

5 

4 

1 1 . Measures and goals 

1.36 

5 

1 

1.29 

4 

1 

12. Retrospective data 

1.73 

5 

1 

1.50 

4 

2 








Total 

27.31 

43 


27.02 

48 


Theoretical Highest Score* 


59 



56 



Maximum possible score: 60 points. 


The Solution: Institutional Change 

It is not enough for tlie administration to reaffirm the analvlical methods and approach to regulatory 
review embodied in Executive Order 12866 because this approach has produced mediocre results. The 
administration and Congress can both take further steps to ensure that all federal agencies conduct high- 
quality analysis of major proposed regulations and seriously consider the results of that analysis when 
they malcc regulatory' decisions; 

1. RegulatosY impact analysis should be required. Currently, Executive Order 12866 requires regulatory 
impact analysis for significant regulations issued by executive agencies. Tlie requirement is only as 
binding as OTRA chooses, or is allowed, to mal<c it. Within the administration, OIRA is supposed to scr\c 
as a regulatory' gatekeeper, ensuring that agencies conduct high-quality regulatory' analysis and consider it 
seriously. OIRA can enforce Executive Order 12866 by returning regulations to agencies for further 
analysis. Yet OIRA has not returned a regulation to an agency since Januaiy' 6, 2009, during the final day's 
of the Bush administration.^' Tlie Mercatus Regulatory Report Card finds that many regulatory analy ses 


http://www.reginfo.gov/public/do/eoReturnLetters. 
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have significant flaws and tlie quality of regulaton- analysis appears to be unchanged since the Bush 
administration. Given those findings, it's curious that not a single regulation has been returned in two 
years. This signals that agencies can expect OIRA to rubber-stamp proposed regulations. 

When evaluating regulations for the Regulatoiy Report Card, we have foimd that when Congress requires 
agencies to consider specific factors such as costs or efficiency, the) usually do so. Congress could 
provide a backstop to OIRA by requiring federal agencies to conduct thorough regulatory impact analysis 
before tliey issue significant regulations. At a minimum, tlie analysis should present coherent tlieones and 
evidence that (1) define the ultimate outcomes of value to the public the regulation is intended to produce, 
(2) identity the systemic problem the regulation solves in order to produce those outcomes, (3) outline a 
wide \'aricty of alternative ways of dealing with tlic problem, and (4) thoroughly assesses the benefits and 
costs of each alternative . 

No administration has sought to apply the regulatory analysis requirements in executive orders to 
independent agencies, such as the Federal Communications Commission, the Commodity Futures Trading 
Commission, or the new Consumer Financial Protection Bureau. Yet the independent agencies issued 94 
major regulations during the past ten years. In some cases, such as the Consumer Product Safety 
Commission and the Securities and Exchange Commission, Congress has seen fit to require some form of 
economic analysis when agencies issue significant regulations. Such requirements should be standardized 
and applied to all independent agencies. In addition, there should be an OIRA-like review mechanism 
before these agencies can issue regulations. 

2 . Regulatory impact analysis should be objective. Reality isn't optional. Tlie goal of regulator)- analysis 
is to produce knowledge about reality that can infonn decisions. Executive Order 12866 requires agencies 
to consider a wide range of regulator)' options. Regulatory' impact analv'sis can reveal the consequences of 
different options decision makers face. Yet all too often, the regulatory impact analysis gets produced 
after key decisions have already been made. The timing also effectively gives the federal government a 
monopoly on producing regulatory- impact analyses and inhibits the public's ability to affect the quality of 
the analysis when it might actually affect agency decisions. 

Both problems could be mitigated if agencies were required to conduct and publish regulatory' impact 
analyses (along with all underlying studies and data) for public comment before the proposed regulation 
is actually written. Agencies would have analysis of regulatory altcmativcs before they chose which 
alternative to pursue. In addition, tlie public would have tlie opportunity to replicate, improve, and 
comment upon the agency 's economic analysis before the agency uses the analy sis to make decisions. 
(This could be considered part of the advance consultation w'ith stalceholders encouraged by section 2 of 
President Obama's new executive order.) 

Interviews with agency economists reveal that they often face pressure to modify' their analysis to support 
decisions that others in the agency-' have already made.’’ One way to promote objective analysis is to 
separate economists from tlie program offices that propose regulations. Economists' work should be 
evaluated by other economists, with compensation and career advancement depending on the quality of 
tlieir analysis — not on whether the analysis supports decisions the agency has already made for other 
reasons. 

3. Regulatoty impact analysis should be used. Use of regulatory- analysis to make decisions about 
proposed regulations is more the exception than the mle. For 2008 and 2009, agency regulatory analysis 
appears to have affected some regulatory decision for only about one-fifth of the proposed regulations. 


Office of Management and Budget, 2010 Report to Congress on the Benefits and Costs of Federal Regulations 
and Unfunded Mandates on State, Local, and Tribal Entities, 

http://vvww.whitehouse.gov/sites/default/files/omb/legislative/reports/2010_Benefit_Cost_Report.pdf. 

Richard Williams, “The Influence of Regulatory Economists in Federal Health and Safety Agencies" (working 
paper, Mercatus Center at George Mason University, July 2008), 

http://mercatus.org/sltes/default/files/publlcatlon/WP0815_Regulatory%20Economlsts.pdf. 
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Either agencies ignore tlie regulator^' analysis in the vast majoritv^ of cases, or they do not explain publicly 
how they have used it. At worst, the analysis often has little effect. At best, there is a significant 
transparency problem: agencies may be using the analysis without disclosing how. 

Either problem could be mitigated if Congress directed agencies to supply such explanations. 

SpecificalK, agencies should be required to explain, when they propose regulations, how the following 
aspects of the regulatory^ analysis influenced their decisions: 

• The ultimate outcomes of value to citizens the regulation is supposed to produce, and tlic 
evidence tliat the regulation will in fact produce Aose outcomes 

• Tlie definition and root cause of tlie systemic problem the regulation seeks to solve 

• The alternative approaches to solving the problem the agency considered 

• The benefits, costs, and net benefits of each alternative 

Agencies should be permitted to consider all aspects of the regulatory analysis when maldng decisions, 
but not required to follow a rigid decision rule. Executive Order 12866 directs agencies to consider, when 
making regulatory' decisions, “incentives for inno\'ation. consistency, predictability, the costs of 
enforcement and compliance (to the government, regulated entities, and tlie public), flexibility', 
distributive impacts, and equity Agencies arc to “propose or adopt a regulation only upon a reasoned 
detennination that the benefits of tlie intended regulation justify its costs.’"' As the executive order 
recognizes, there arc some very-' good reasons that agencies should not always be required to pick the 
regulatoiy-' option that maximizes the difference between monetized benefits and monetized costs. 
Uiiquantified values or other important public purposes may in some cases trump benefit-cost 
calculations. 

In some cases, however, legislation prohibits agencies from considering certain effects of regulation, such 
as costs. Tliere is much less justification for forcing some agencies to "fly blind” by prohibiting them 
from even considering some types of information from the regulatory' impact analysis when making 
important decisions. Such proliibitions mandate ignorance. 

Conclusion 

Regulations produce hundreds of billions of dollars worth of benefits and costs. Yet all too often, the 
analysis required to make infonned decisions about proposed regulations fails to satisfy the requirements 
adopted by presidents from both political parties during the past three decades. The problem is not 
political; it persists under both Democratic and Republican administrations. Tlie problem is institutional, 
and only' institutional change whll solve it. Regulatory' impact analysis should be legislatively required for 
all agencies. Congress can improve tlie objectivity' of tlie analy'sis by requiring agencies to perform and 
publish the analysis, the underlying data, and the underlying source documents prior to writing 
regulations. Finally, Congress can encourage use of the analysis by requiring agencies to explain how key 
aspects of the analysis affected their decisions. 


Executive Order 12866, secs. 1(5) and 1(6). 
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Assessing the Quality of Regulatory Analysis: 

A New Evaluation and Data Set for Policy Research 


Abstract 

Congress and the executive branch have attempted to improve the quality of regulatory decisions 
by adopting laws and executive orders that require agencies to analyze benefits and costs of their 
decision options. This paper assesses the quality and use of regulatory analysis accompanying 
every economically significant regulation proposed by executive-branch regulatory agencies in 
2008 and 2009. It considers all analysis relevant to the topics covered by Executive Order 12866 
that appears in the Regulatory Impact Analysis document or elsewhere in the Federal Register 
notice that proposes the rule. 

Our research team used a six-point qualitative scale to evaluate each regulation on 12 criteria 
grouped into three categories: (1) Openness: How easily can a reasonably intelligent, interested 
citizen find the analysis, understand it, and verify the underlying assumptions and data? (2) 
Analysis: How well does the analysis define and measure the outcomes the regulation seeks to 
accomplish, define the systemic problem the regulation seeks to solve, identify and assess 
alternatives, and evaluate costs and benefits'i^; and (3) Use: How much did the analysis affect 
decisions in the proposed rule, and what provisions did the agency make for tracking the rule's 
effectiveness in the future? 

We find that the quality of regulatory analysis is generally low, varies widely, and did not change 
much with the change of administrations between 2008 and 2009. The principal improvements 
across all regulations occurred on the Openness criteria. Budget or “transfer” regulations, which 
define how the federal government will spend money or collect revenues, have much lower- 
quality analysis than other regulations. Use of analysis is correlated with its quality, and use of 
analysis fell in 2009 after controlling for the quality of the analysis. Regulations implementing 
Recovery Act spending programs have better provisions for retrospective analysis than other 
transfer regulations. 


Keywords: regulatory impact analysis, benefit-cost analysis, regulatory review, regulation 
JEL categories: D6 1 , D73, D78, HU, H83, K23, L5 1 , P 1 6 
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Introduction 

For nearly four decades, presidential administrations have required executive-branch 
agencies to conduct some type of economic impact analysis when they issue major regulations. 
Since 1993, President Clinton’s Executive Order 12866 has laid out the fundamental analytical 
steps agencies must take. The very first section of the executive order states that agencies must 
identify the problem they are trying to address and assess its significance, examine a wide range 
of alternatives to solve the problem, assess the costs and benefits of the alternatives, and choose 
to regulate only when the benefits Justify the costs. Analytical requirements are especially 
rigorous for "economically significant” regulations, defined as regulations that “have an annual 
effect on the economy of $100 million or more or adversely affect in a material way the 
economy, a sector of the economy, productivity, competition. Jobs, the environment, public 
health or safety, or state, local or tribal government or communities” (EO 12866, Sec. 2(f)(1)). 
Office of Management and Budget (0MB) Circular A-4, issued in September 2003, offered more 
detailed guidance on “best practices” in regulatory analysis (0MB 2003). 

Despite executive orders and detailed guidance, the quality of agencies’ regulatory 
analysis has been inconsistent at best: 

• Several studies compared agencies’ ex-ante predictions of regulatory benefits and costs 
with ex-post estimate of actual benefits and costs (Harrington et. al. 2000, 0MB 2005, 
Harrington 2006). These studies found that, in the past, ex-ante estimates tended to 
overestimate both benefits and costs. 

• In a series of papers, Robert Hahn developed and applied a yes/no checklist to evaluate 
whether agencies’ Regulatory Impact Analyses have included a series of major elements 
that 0MB expects them to include. The evaluations focused on final regulations issued by 
health, safety, and environmental agencies (Hahn and Dudley 2007, Hahn et. al. 1990, 
Hahn and Litan 2005, Hahn, Lutter, and Viscusi 2000). Surveying the evidence, Hahn 
and Tetlock (2008, 82-83) conclude that economic analysis has not had much impact, 
and the general quality of regulatory analysis is low. “Nonetheless,” they note, “in a 
world where regulatory impacts are frequently measured in the billions of dollars, 
margins matter. Thus, economists should pay more attention to how economic analysis 
can contribute to improving benefits and costs on the margin.” 

• Belcore and Ellig (2008) employed a qualitative scoring approach to assess the quality of 
regulatory analysis at the Department of Homeland Security during its first five years; 
they conclude these analyses have been seriously incomplete but improved over time. 

Most recently, Ellig and McLaughlin (2010) developed a 12-point qualitative framework 
to assess both the quality and use of regulatory analysis in federal agencies. They evaluated the 
quality and use of regulatory analyses of “economically significant” rules that were reviewed by 
OMB’s Office of Information and Regulatory Affairs (OIRA) in 2008 and proposed in the 
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Federal Register. ’ The evaluation criteria are drawn from Executive Order 1 2866, 0MB Circular 
A-4, and pre-existing scholarship on regulatory scorecards.^ Ellig and McLaughlin found that the 
average quality of the 2008 regulatory analyses is low, both the quality and use of regulatory 
analysis vary widely, and there are significant opportunities for improvement through the 
diffusion of best practices. They also found that better analyses are more likely to be used in 
agency decisions, but only one-fifth of the regulatory analyses in 2008 appeared to have any 
effect on regulatory decisions (based on information agencies supplied in the preamble). 

This study utilizes the Ellig and McLaughlin method to evaluate the quality and use of 
regulatory analysis for economically significant regulations proposed by executive-branch 
agencies in 2009. This is of interest for several reasons. First, a comparison of 2008 and 2009 
would help identify whether the change of presidential administrations had any effect on the 
quality or use of regulatory analysis. Second, the Obama administration proposed in February 
2009 to revise Executive Order 12866 (0MB 2009a); evaluating the quality and use of 
regulatory analysis in the Obama administration prior to the revision establishes a baseline to 
gauge the effects of any changes. Third, extending the evaluation to 2009 and subsequent years 
builds a larger data set, which may allow us to draw more reliable general inferences about the 
relative quality of analysis at different agencies or for different types of regulations. 

Our principal findings include; 

Quality is mostly unchanged in 2009. The average score for regulations proposed in 2008 and 
2009 was virtually the same — 27 points out of a possible 60. The most significant improvements 
occurred on Openness criteria, such as online accessibility of regulatory analyses and clarity. On 
average, explanations of how regulatory costs affect prices of goods and services also improved. 
Very modest improvements occurred in evidence of regulatory benefits and analysis of the 
distribution of benefits. 

Analysis is less-widely used in 2009. Higher-quality analysis is more likely to be used in 
regulatory decisions. But for any given level of quality, regulatory agencies were less likely to 
use the analysis in 2009 than in 2008. This change is disturbing, because one of the most 
important reasons for doing regulatory analysis is so that decision makers can somehow use it to 
make better decisions. Of course, good regulatory analysis is also important for reviewers (like 
0MB) and stakeholders. 

Quality Is generally low. Tn both years, the average score is less than half of the possible 60 
points. The highest-scoring regulation in 2008 earned 43 out of 60 possible points, equivalent to 
a grade of C. The highest-scoring regulation in 2009 earned 48 out of 60 possible points, 
equivalent to a B-. 


' Economically significant regulations require an extensive Regulatoiv Impact Analysis (RI A) that assesses the 
need, effeetiveness, benefits, eosts, and alternatives for the proposed regulation. (EO 1 2866 See. 6(a)(3)(C)) 

■ The qualilalive evaUialion melhod is based on ihe Mercalus Center's Performance Report Scorecard, a iO-year 
project llial assessed the quality of federal agencies' amiual performance reports required under the Govermnenl 
Peifoiinance and Results Act of 1996. For the most recent results, see McTigue et. al. (2009). 
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Diffusion of best practices could generate substantial improvement. In 2009, scores ranged 
from a high of 48 points to a low of just 3 points. In 2008, scores ranged from a high of 43 points 
to a low of 7 points. For each of our 12 criteria, at least one regulation earned the highest 
possible score of 5. But for 1 1 of our 12 criteria, less than a handful of regulations receive a 5. 
The fact that the highest-scoring regulation in 2009 resulted from collaboration between two 
agencies also suggests wider sharing of best practices can improve regulatory analysis. 

Transfer regulations have worse analysis. Budget or “transfer” regulations, which determine 
how the federal government will spend or collect money, receive much lower scores. On 
average, transfer regulations received only 17 points in 2008 and 20 points in 2009, compared to 
an average of 32-34 points for non-transfer regulations. 

Greatest strength: Accessibility on the Internet. Scores on this criterion averaged 4.06 out of 5 
possible points in 2009 and 3.53 out of 5 possible points in 2008. These far exceeded average 
scores on any other evaluation criterion. 

Greatest weaknesses: Retrospective analysis and identification of systemic problem. Few 

regulations or analyses set goals, establish measures, or provide for data gathering to assess the 
effects of the regulation after it is implemented. Few analyses provide a coherent theory and 
empirical evidence of a market failure, government failure, or other systemic problem the 
regulation is supposed to solve. 
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1. Evaluation Protocol 

We evaluated the quality and use of regulatory analysis using 12 criteria grouped into 
three categories — Openness, Analysis, and Use: 

1 . Openness: How easily can a reasonably intelligent, interested citizen find the analysis, 
understand it, and verify the underlying assumptions and data? 

2. Analysis: How well does the analysis define and measure the outcomes or benefits the 
regulation seeks to accomplish, define the systemic problem the regulation seeks to solve, 
identify and assess alternatives, and evaluate costs and benefits? 

3. Use: How much did the analysis affect decisions in the proposed rule, and what 
provisions did the agency make for tracking the rule’s effectiveness in the future? 

Figure 1 lists the 12 criteria. Appendix 1 provides additional detail on the kinds of 
questions considered under each criterion. For a more extensive explanation and justification of 
this evaluation method, see Ellig and McLaughlin (2010). Individual ‘"Report Cards” showing all 
scores and scoring notes for each regulation are available at wwv.M.oercatus.OTvVreportcard . 

Ten of the 12 evaluation criteria closely parallel the Regulatory Impact Analysis checklist 
released by the Obama administration on November 3, 2010 (0MB 2010). This is not surprising, 
since both the administration’s checklist and the Mercatus evaluation criteria are based on 
Executive Order 12866 and 0MB Circular A-4. Appendix 2 presents a crosswalk chart 
comparing the 0MB checklist with the 12 criteria used in this paper. 

The principal Mercatus evaluation criteria not mentioned in the Obama administration’s 
checklist are two criteria that assess whether the agency provided for retrospective analysis of the 
regulations’ actual effects after it is adopted: criterion 1 1 (Measures and Goals) and criterion 12 
(Retrospective Data). Although ex post, retrospective analysis has not received as much attention 
as ex ante analysis of proposed regulations, section 5 of Executive Order 12866 states that 
agencies should conduct retrospective analysis. 0MB (2005) has recommended it repeatedly; 
most recently, 0MB (2009b, 45) stated, “[W]e recommend that serious consideration be given to 
finding ways to employ retrospective analysis more regularly, in order to ensure that rules are 
appropriate, and to expand, reduce, or repeal them in accordance with what has been learned.” 
The Government Performance and Results Act arguably requires retrospective analysis of 
regulations (Brito and Ellig 2009). It is a major area of regulatory analysis where the United 
States lags other industrialized nations (OECD 2009, 92). 
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Figure 1: Regulatory Analysis Assessment Criteria 

Openness 

1 . Accessibility: How easily were the Regulatory Impact Analysis, the proposed rule, and 
any supplementary materials found online? 

2. Data Documentation: How verifiable are the data used in the analysis? 

3. Model Documentation: How verifiable are the models and assumptions used in the 
analysis? 

4. Clarity: Was the analysis comprehensible to an informed layperson? 

Analysis 

5. Outcomes: How well does the analysis identify the desired benefits or other outcomes 
and demonstrate that the regulation will achieve them? 

6. Systemic Problem: How well does the analysis identify and demonstrate the existence of 
a market failure or other systemic problem the regulation is supposed to solve? 

7. Alternatives: How well does the analysis assess the effectiveness of alternative 
approaches? 

8. Benefit-Cost Analysis: How well does the analysis assess costs and compare them with 
benefits? 

Use 

9. Some Use of Analysis: Does the preamble to the proposed rule or the Regulatory Impact 
Analysis present evidence that the agency used the analysis'^ 

10. Cognizance of Net Benefits: Did the agency maximize net benefits or explain why it 
chose another option‘d 

11. Measures and Goals: Does the proposed rule establish measures and goals that can be 
used to track the regulation’s results in the ftiture? 

12. Retrospective Data: Did the agency indicate what data it will use to assess the 
regulation’s performance in the future and establish provisions for doing so? 


Scoring Standards 

For each criterion, the evaluators assigned a score ranging from 0 (no useftil content) to 5 
(comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to 
earn between 0 and 60 points. In general, the research team used the guidelines in table 1 for 
scoring. Because the Analysis criteria involve so many discrete aspects of regulatory analysis, 
we developed a series of sub-questions for each of the four Analysis criteria and awarded a 0-5 
score for each sub-question. These scores were then averaged to calculate the score for the 
individual criterion. 
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Table 1: Whal Do the Scores Mean? 

of all or hIiuum all aspects, witli one or iiiorr ’Miesi 



il^rnMuiaMv ib«iroii{’ti uf most aspefU mnl/pr sWimi ai least war *'bv%t 

3 Reasonably thorough analysis of some aspects 


i! 


ome'rdtviuit diicmrioii irift <onie documoiuitoii of iiulysb 


rri'luiii*iorj Bt:i[rfnFiit with lUUmplanatidii or tlficujnenlaiion 


l.jnk- wr nn iclevjini cunlrni 


Caveats anil QualiTicntions 

At the outset of this project, we had to address a seemingly simple question What counts 
as a "regulatory analysis"? Most previous research focuses on the document required by OMB 
that is explicitly named the "Regulatory Impact Analysis” (Hahn and Dudley 2007, Hahn et al 
1990. Hahn and Litan 2005, Hahn, Lutter. and Viscusi 2000) We adopted a broader definition 
that includes the entire preamble to the proposed rule, the freestanding document or section of 
the preamble labeled Regulatory Impact Analysis, and additional "technical support documents” 
that sometimes accompany a Regulatory Impact Analysis Since different agencies organize their 
material in different ways, this approach helped ensure that we were lair to all agencies and 
included all material relevant to the topics a good regulatory analysis is supposed to address We 
also needed to read the entire preamble to assess whether the agency used the results of the 
regulatory analysis or made provisions to conduct retrospective analysis in the future 

Given resource constraints, any ev aluation project like this faces a fundamental choice 
between breadth and depth of the assessment We assess whether the Regulatory Impact Analysis 
and preamble to the proposed rule make a reasonable effort at covering the major elements of 
regulatory analysis Commenters on earlier versions of this paper who have detailed knowledge 
of particular regulations have usually told us that our evaluations seem loo lenient Others with 
more specialized knowledge will likely have additional important critiques of individual 
regulations, especially related to the quality, completeness or use of the underlying science We 
have opted for less depth in favor of greater breadth To the best of our knowledge, this is the 
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most-detailed assessment of the quality of regulatory analysis for all economically significant 
regulations proposed in a two-year period. 

Finally, we caution the reader about drawing direct policy conclusions about particular 
regulations based on our analysis. Criteria 1-8 only evaluate the quality of regulatory analysis. 
We do not evaluate whether the proposed rule is economically efficient, fair, or otherwise good 
public policy. 

The same caveat applies to the Use criteria. Criteria 9 and 10 assess the extent to which 
analysis of the regulation’s outcomes or benefits, the systemic problem, the alternatives, and 
costs informed the agency’s decisions about the regulation. On these criteria, we took great pains 
to avoid imposing the value judgment economists often make: that the agency should choose the 
most economically efficient alternative, as determined by a comparison of quantified benefits 
and costs. If an agency used some analysis of a regulation’s benefits to make decisions, even if it 
did not consider costs or efficiency, it could receive some points on criterion 9. Similarly, if an 
agency demonstrated that it was fully cognizant of the net benefits of alternatives, but explicitly 
rejected the alternative with the greatest net benefits in favor of some other alternative for clearly 
articulated reasons, it could receive points on criterion 10. As a result, an agency can earn points 
on these two criteria even in cases where it is prohibited by law from considering costs, such as 
the EPA’s national ambient air quality standards We believe this approach is consistent with the 
spirit of Executive Order 12866 (sec. I), which identifies multiple factors in addition to 
efficiency that are supposed to guide agency decisions: “[I]n choosing among regulatory 
approaches, agencies should select those approaches that maximize net benefits (including 
potential economic, environmental, public health and safety, and other advantages; distributive 
impacts; and equity), unless a statute requires another regulatory approach.” 

Criteria 11 and 12 assess the extent to which the agency demonstrated its willingness to 
evaluate the regulation’s actual effects in the future. Ideally, agencies would articulate goals, 
measures, and data that they could use to assess both realized benefits and costs, thus assessing 
the regulation’s economic efficiency. In practice, so few regulations include any provisions for 
retrospective analysis that the handful of high scores occur in cases where agencies have at least 
identified goals, measures, and data that could be used to assess the regulation’s effectiveness. 

Improving the transparency of regulatory documents and the quality of regulatory 
analysis are necessary but not sufficient to improve public policy. Nevertheless, stakeholders or 
the agencies themselves may find these analyses useful as a starting point for identifying 
weaknesses in agency analyses. For example, if an agency has identified only one or two closely 
related regulatory alternatives, stakeholders may be able to identify additional alternatives that 
may accomplish the goal at a lower cost. 
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2. Results for 2009 

2.1 Best and Worst Analyses 

Table 2 lists all 42 economically significant proposed regulations for 2009. The best 
analysis was for the combined Environmental Protection Agency-Department of Transportation 
regulation on greenhouse gases from light-duty vehicles and Corporate Average Fuel Economy 
(CAFE) standards. This regulation received the highest total score (48 points) as well as the 
highest Analysis score (18 points). The two agencies collaborated on developing the regulation 
and the analysis. The regulatory analysis discusses the “conundrum” associated with the 
identified market failure. The agencies recognize that their estimates of the private benefits of 
increased fuel efficiency outweigh private costs, yet consumers do not voluntarily purchase as 
many fuel-efficient cars as economic rationality would suggest. This sort of disclosure should 
prove invaluable to stakeholders who wish to comment more extensively on the merits of the 
rule that requires increases in fuel efficiency. The result suggests that more extensive sharing of 
best practices could improve the quality of regulatory analysis. This regulation received a score 
six points higher than the next-best regulation in 2009 and five points higher than DOT’s CAFE 
regulation in 2008. 

Capturing second place in 2009 are three energy-efficiency regulations from the 
Department of Energy and the Department of Homeland Security's regulation limiting 
concentrations of live organisms permitted in discharged ballast water from ships. 

The three worst analyses came from the Department of Education (General and Non- 
Loan Programmatic Issues, 14 points) and the Department of Energy (Weatherization 
Assistance, 10 points; Loan Guarantees for Projects that Employ Innovative Technologies, 5 
points). Like most of the low-ranking regulations, all three of these are budget or “transfer” 
regulations. Transfer regulations, italicized in table 2, outline how the federal government will 
spend money, set fees, or administer spending programs. Most of these regulations score poorly, 
continuing atrend observed in 2008 (Ellig and McLaughlin 2010, 14-15). 

The best analysis in 2009 received 48 points, or 80 percent of the ma.ximum possible 
score. The worst received just five points (8 percent). The range of scores widened compared to 
2008. In 2008, scores ranged from seven points to 43 points. If these were student papers, the 
best one in 2009 would have received a B-, and the best one in 2008 would have received a C. 

2.2 Summary Statistics 

Table 3 summarizes average total scores and scores on the three categories of criteria for 
2008 and 2009. The average score in 2009 was 27.02 points out of a possible 60, or 45 percent. 
The average for 2008 was 27.3 1, virtually the same. The very low t-statistic indicates that the 
difference is not statistically significant; for all practical purposes, the averages are the same.'^ 


In plain Englisli, llial means the lolal scores for 2008 and 2009 are like Uvo sels of ping pong balls pulled at 
random out of ihe same bucket; anv difference in the ax'erages is random chance. There is tikel> no difference at all 
between the total scores for the tw'o years. 
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Table 2; Scores for 2009 Proposed Regiilalions 


Proposed Rule 

GfetnoouM G*t«t fremli|ht'OutvVthclci 

EnerfyCoraervitioniSmill Dsctric Motors 

Enerfv Cffocncy Stondords for Commerdol ClOlha Wothors 

Entrtv EK<fcncySUndards for Pool He«tm tic 

lMn( Organksm in Snips' Water Dtsenarged tp Waters 

Nutrition LaOcting of Single Ingrodieni Products 

Till* V Greenhous* Gaa TaMrirtg Rut* 

Emissions From New Marine Compression- Ignition Engines 
Portland Cement NESHAP 
Greennotoe Gas Mandatory Reporting Ru)e 
Migratory Bird Hunting 

Emission Standards, Reciprocating internal ComPustlon Engines 

fnd UOpc Rertof Ourasr PrapeeUw PoytmnI Syirem 
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DOE 
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DOE 

40 

14 

14 
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DOE 
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14 

14 
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16 
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Some slight shifts in scores may have occurred in two of Ihc categories between ;008 
and 2009 The average Analysis score was largely unchanged The average Openness score 
increased by about one point-from 1 1 ,04 in 2008 to 12 in 2009 The average Use score fell by 
about a point, from 7.73 in 2008 to 6.64 in 2009 These differences are statistically significant at 
the 85 percent confidence level This is suggestive, but not nearly as strong an indicator as the 95 
percent conftdence level economists normally use as the standard to infer a likely relationship 
Based on this comparison of averages for all kinds of regulations, perhaps the transparency of 
regulatory analysis increased in 2009. and actual use to make decisions may have decreased, but 
the difference is not clear enough to tell for sure 

Figure 2 shows that the distribution of scores was roughly the same in both years The 
only differences are that the joint DOT/EPA regulation received a score of 48 in 2009, and 
several more regulations in 2008 received scores in the 36-47 range 

Table 3: .Average Scores. 2008 vs. 2009 



2008 

2009 




en=45J 

(n=42) 

Change 

T-stal. 

Total Score 

27.31 

27.02 

-0 29 

0.14 

Openness 

11 04 

12.00 

096 

1 46 

Analysis 

8 53 

8 38 

-0 15 

0 16 

Use 

7.73 

664 

-1.09 

1 48 


Maximum possible total score - 60 Maximum possible score on each category ^ 20 


Figure 2: Distribution of Scores 
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2.3 Average Scores by Criterion 

Table 4 shows the average score for each criterion in 2008 and 2009. For each criterion, 
at least one regulation earned the highest possible score of 5 in most cases. Best practices, 
however, are not widely shared. The “# Earning Highest Score” column demonstrates that, 
except for Availability, very few regulations earn a score of 5 on any individual criterion. The 
“Theoretical Highest Score” is the score a hypothetical regulation could have earned if it had 
incorporated all of the best practices identified that year. For 2009, the highest-scoring regulation 
is much closer to the theoretical highest score than in 2008, 


Table 4: Scores by Criterion 


Criterion 

2008 

Average 

Score 

2008 

Highest 

Score 

2008# 

Earning 

Highest 

Score 

2009 

Average 

Score 

2009 

Highest 

Score 

2009# 

Earning 

Highest 

Score 

1. Accessibility 

3.53 

5 

12 

4.06 

5 

14 

2. Data Documentation 

2.24 

5 

1 

2.50 

5 

5 

3. Model Documentation 

2.33 

5 

3 

2.62 

5 

1 

4. Clarity 

2.93 

5 

3 

2.83 

4 

10 

5. Outcome Definition 

2.36 

5 

2 

2.38 

5 

1 

6. Systemic Problem 

1.80 

5 

1 

1.60 

4 

4 

7. Alternatives 

2.29 

5 

1 

2.21 

5 

1 

8. Benefit-Cost Analysis 

2.09 

4 

3 

2.19 

5 

1 

9. Some Use of Analysis 

2.44 

5 

2 

2.24 

5 

1 

10. Considered Net Benefits 

2.20 

5 

2 

1.62 

5 

4 

11. Measures and Goals 

1.36 

5 

1 

1.29 

4 

1 

12. Retrospective Data 

1.73 

5 

1 

1.50 

4 

2 








Total 

27.31 

43 


27.02 

48 


Theoretical Highest Score* 


59 



56 



Very few of the score changes between 2008 and 2009 are statistically significant.'^ 
Moreover, changes in averages for some criteria appear to be driven by the changing mix of 
regulations rather than an actual change in the quality of agencies’ analysis. An accurate 
assessment of changes, therefore, requires separate consideration of transfer and non-transfer 
regulations.'^ 


' SLiminary slalislics for all criteria, and the sub-questions for criteria 5-8, are in appendix 3. 

^ Statistically signihcant changes in averages for the entire set of regulations, without distinguisliing between 
transfer and iion-Traiisfer regulations, are in appendix 4. 
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2.4 Transfer vs. Non-Transfer Regulations 

Several previous studies using 2008 data, as well as table 2, demonstrate that the quality 
and use of analysis for transfer regulations is well below the quality and use of analysis for non- 
transfer regulations (Ellig and McLaughlin 2010, McLaughlin and Ellig 2010). Indeed, 0MB 
(2008, 12-17) observes that although transfer regulations generate social costs via mandates, 
prohibitions, and price distortions, agencies do not usually estimate the social benefits and costs 
of transfer regulations. 

Table 5 confirms that the quality and use of analysis for transfer regulations is much 
lower in both 2008 and 2009. In 2008, for example, the average total score for transfer 
regulations (17 points) is 47 percent below the average score for non-transfer regulations (32 
points). Similarly, in 2009 the average total score for transfer regulations (21 points) is 40 
percent below the average total score for non-transfer regulations (34 points). These differences 
occur for Openness, Analysis, and Use. Openness has the smallest gap, but even there, transfer 
regulations score 20-30 percent below non-transfer regulations. 


Table 5: Transfer vs. Non-Transfer Regulations, Average Scores 



Transfer 2008 

(n=15) 

Non-Transfer 2008 

(n=30) 

Difference 

T-stat. 

Total Score 

17.07 

32.43 

15.37 

8.03 

Openness 

8.6 

12.27 

3.67 

4.16 

Analysis 

3.53 

11.03 

8.53 

8.71 

Use 

4.93 

9.13 

4.20 

4.99 


Transfer 2009 

(n=22) 

Non-Transfer 2009 

(n=20) 

Difference 

T-stat. 

Total Score 

20.54 

34.15 

13.65 

6.84 

Openness 

10.5 

13,65 

3.15 

4.32 

Analysis 

4.91 

12,20 

7.29 

8.9 

Use 

5.14 

8.3 

3.16 

3.18 


All differences are statistically significant at greater than the 99 percent level of confidence. 
Maximum possible total score = 60. Maximum possible score on each category = 20. 


Because transfer regulations generally receive lower scores, a shift in the mix of transfer 
vs. non-transfer regulations could affect changes in average scores from one year to the next. In 
2008, there were 15 proposed economically significant transfer regulations, accounting for 33 
percent of proposed economically significant regulations. In 2009, there were 22 proposed 
economically significant transfer regulations, accounting for 52 percent of proposed 
economically significant regulations. The increase mostly reflects five regulations proposed in 
2009 that implemented provisions of the American Recovery and Reinvestment Act. Thus, one 
might expect that the average quality and use of regulatory analysis would be lower in 2009 than 
in 2008 simply because more transfer regulations were proposed in 2009. 
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Table 6: Score Changes on Individual Criteria and Questions, Transfer vs. Non-Transfer 
Regulations 



2008 

(n=30) 

2009 

(n=20) 

Change 

T-stat. 

Non-Transfer Regulations 





Total Score 

32,43 

34,15 

1,72 

0,94 






Openness 

12,27 

13,65 

1,38 

1,91* 

Criterion 1 - Availability 

3,30 

3,95 

0.65 

1.69* 

Criterion 2 - Data Documentation 

2,63 

3,15 

0,52 

1,66* 

Criterion 3 - Theory and Model Documentation 

2,83 

3.30 

0.47 

1.49 






Analysis 

11,03 

12.20 

1.17 

0.20 

Criterion 5 - Outcomes 

3.10 

3.55 

0,45 

1.63 

Question 5D - Evidence Regulation Will Affect Outcome 

2,40 

3.15 

0,75 

-X- 

00 

00 

Criterion 8 - Cost-Benefit Analysis 

2,60 

3,10 

0,5 

2,15** 

Question 8C - Effects on Prices of Goods and Services 

1,70 

3.30 

1,60 

3,91*** 

Question 8G - Calculates Cost-Effectiveness 

1,43 

2,35 

0,92 

2.35** 

Question 81- Incidence of Benefits 

2,07 

2.95 

0.88 

2.33** 






Use 

9,13 

8.3 

-0.83 

0.35 






Transfer Regulations 





Total Score 

17,07 

20.55 

3.48 

1.70* 






Openness 

8,60 

10,50 

1,90 

2.11** 

Criterion 3 - Theory and Model Documentation 

1,33 

2,00 

,67 

-X- 

00 

00 

Criterion 4 - Clarity 

1.80 

2.45 

.65 

2.37** 






Analysis 





Criterion 5 - Outcomes 

0,87 

1,31 

0,45 

1,61 

Question 5A - Articulate Desired Outcome 

1,80 

2.45 

0.65 

1.52 

Question 5D - Evidence Regulation Will Affect Outcome 

0.20 

1.00 

0,80 

2,86*** 

Criterion 6 - Systemic Problem 

0,60 

1.00 

0.40 

1.79* 

Question 6B - Coherent Theory of Systemic Problem 

0,47 

0,86 

0,40 

1,64 

Question 7A - List Alternatiyes 

1,07 

1,91 

0,84 

2,18** 

Criterion 8 - Cost-Benefit Analysis 

1,07 

1,36 

0,30 

1,51 






Use 

4,93 

5.14 

0.20 

0.83 


Statistical significance; *90 percent ** 95 percent 
Maximum possible score on individual criteria or questions = 5. 
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Table 6 shows changes in mean scores calculated separately for transfer and non-transfer 
regulations. We report statistics for individual criteria or questions only when the differences 
approach statistical significance. 

For non-transfer regulations, there are very few improvements. Average Openness scores 
improved from 12.27 points to 13.65 points. The difference is almost statistically significant at 
the 95 percent level. Within the Analysis category, there is weak evidence of improvement on 
criterion 5 (Outcomes), largely because agencies provided more evidence that the regulation will 
accomplish the intended outcomes. Criterion 8 (Cost-Benefit Analysis) also saw improvement 
due to better scores on three questions: question 8C (Effects on Prices of Goods and Services), 
question 8G (Evaluation of Cost-Effectiveness) and question 81 (Incidence of Benefits). These 
changes are consistent with the administration’s goals of improving the transparency of the 
regulatory process, identifying benefits of regulation, and expanding the focus on distributional 
issues. We caution, however, that the changes are quite small, and the improvements under the 
Analysis category mostly just move the average scores closer to 3. 

Transfer regulations show slightly more improvement than non-transfer regulations. The 
average Openness score improved, largely due to increases in scores on criterion 3 (Theory and 
Model Documentation) and criterion 4 (Clarity). The improvement on criterion 4 is actually 
significant at the 98 percent level. All four Analysis criteria saw higher average scores in 2009 
than in 2008. However, all of these scores remained well below 2 in 2009. This indicates only 
that more analyses presented a small amount of discussion or evidence relevant to these criteria 
instead of saying nothing. While these improvements are certainly welcome, the low levels of the 
scores indicate that analysis of transfer regulations has a long way to go before it is as good as 
the analysis of non-transfer regulations. 

We draw the following conclusions from this breakdown between transfer and non- 
transfer regulations: 

• The only category of criteria that appears to have improved for both transfer and non- 
transfer regulations is Openness. 

• The few improvements in the Analysis criteria for non-transfer regulations seem 
consistent with the Obama administration’s regulatory priorities. 

• Improvements in some of the Analysis criteria for transfer regulations largely refect the 
presence of some content or assertions where previously there were none. 

• Regulators made little commitment to retrospective analysis of regulations proposed in 
either year. 

2.5 Total Scores by Agency 

Another way to control for factors that might affect the average quality or use of 
regulatory analysis is to break scores down by agency. Some agencies may do abetter job of 
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analysis than others. Some may tackle analytical problems that are inherently more difficult. Yet 
others may have different mixes of transfer regulations and non-transfer regulations. Table 7 
presents average scores by agency for 2008 and 2009, with and without transfer regulations. 

When all regulations are included, five agencies increased their average total scores in 
2009, and five agencies reduced their average total scores. When transfer regulations are 
excluded, four agencies increased their average total scores in 2009, and four agencies reduced 
their average total scores. Given that most agencies proposed small numbers of economically 
significant regulations, few agencies proposed comparable numbers of economically significant 
regulations in both years, and six agencies proposed economically significant regulations only in 
2008, it is difficult to infer any general pattern of improvement or deterioration from these 
results. 


However, it is clear that the presence or absence of transfer regulations in a given year 
has a big effect on some agencies' scores. Scores for the Departments of Energy, Homeland 
Security, Transportation, and Health and Human Services climb noticeably in one or both years 
when transfer regulations are excluded. Omitting transfer regulations, Energy and Homeland 
Security leapfrog Agriculture, EPA, and Interior in the 2009 rankings, and HHS edges past 
Labor. 
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Table 7: Average Total Scores by Agency 


All Regulations 

2009 

Average 

Score 

# of 

Regulations 

2008 

Average 

Score 

# of 

Regulations 

2008-09 

Change 

Joint DOT/EPA 

48.0 

1 

NA 

0 

NA 

USDA 

38.0 

1 

28.0 

1 

+ 10.0 

Interior 

34.0 

1 

27.3 

4 

+6.7 

EPA 

32.5 

9 

39.5 

2 

-7.0 

DHS 

30.0 

2 

38.0 

2 

-8.0 

Energy 

27.4 

5 

27.0 

1 

+0.4 

DOT 

24.7 

3 

32.3 

6 

-7.6 

Labor 

24.0 

1 

34.1 

6 

-10.1 

HHS 

23.6 

12 

20.7 

11 

+2.9 

Education 

22.0 

5 

22.0 

2 

0 

HUD 

18.0 

1 

41.0 

1 

-23.0 

Veterans 

17.0 

1 

10.0 

1 

+7.0 

Justice 


0 

35.0 

3 

NA 

Treasury 


0 

27.0 

1 

NA 

Fed Acquisition 


0 

24.0 

1 

NA 

State 


0 

13.0 

1 

NA 

Defense 


0 

12.0 

1 

NA 

SSA 


0 

7.0 

1 

NA 







Non-Transfer 

Regulations 

2009 

Score 

#of 

Regulations 

2008 

Score 

#of 

Regulations 

2008-09 

Change 

Joint DOT/EPA 

48.0 

1 

NA 

0 

NA 

Energy 

40.7 

3 

27.0 

1 

+ 13.7 

DHS 

40.0 

1 

38.0 

1 

+2.0 

USDA 

38.0 

1 

28.0 

1 

+10.0 

EPA 

32.5 

9 

39.5 

2 

-7.0 

Interior 

34.0 

1 

27.3 

4 

+6.7 

DOT 

29.0 

2 

32.3 

6 

—3.3 

HHS 

28.0 

1 

29.0 

2 

-1.0 

Labor 

24.0 

1 

34.1 

6 

-lO.I 

HUD 


0 

41.0 

1 

NA 

Justice 


0 

35.0 

3 

NA 

Treasury 


0 

27.0 

1 

NA 

Federal Acquisition 


0 

24.0 

1 

NA 


Maximum possible average total score = 60. 
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5. Use of Analysis 

Previous research found that use of the analysis was positively correlated with the quality 
of the analysis in 2008. Scores on criteria 9-12, which evaluate use of analysis, are positively 
correlated with the Analysis score and overall quality, defined as the sum of the Openness and 
Analysis scores, criteria 1-8 (Ellig and McLaughlin 2010), An additional year gives us a larger 
data set to test whether this relationship still held and whether it changed in 2009. 

5.1 Total Use Score 

Table 8 shows the results from regressing the Use score on the Quality score, along with 
several control variables. A one point increase in the Quality score is associated with a 0.25-0.31 
point increase in the Use score, and this correlation is highly statistically significant. The result 
also seems quantitatively significant. The standard deviation of Quality is 6.86; a one-standard- 
deviation change in Quality implies about a two-point change in Use. Given that the mean Use 
score is 7.21, variation in Quality seems to explain a great deal of the variation in Use.^' 

The Year 2008 dummy tests whether Use scores tend to be different in 2008 and 2009, Tt 
shows that Use is about 1.3 points higher in 2008, after controlling for Quality. This result 
indicates a 1.3-point shift in the intercept of the regression equation. One might also speculate 
that the slope of the line might be different in the two years. When we run the same regressions 
using Quality x Year as an explanatory variable instead of the year dummy, we get roughly the 
same results with a bit worse statistical fit.^ 

The year appears to make a big difference, considering that the mean Use score is only 
7.21 and its standard deviation is 3.45. However, it would be a mistake to portray the first year of 
the Obama administration as a retreat from stellar use of analysis in the Bush administration. 
Figure 3 shows the distribution of Use scores in 2008 and 2009. Neither year shows more than 
middling use of analysis. The principal difference is that the middle class shrinks in 2009, with 
more regulations that either fail to use the analysis or make only a passing reference to it. 

Models 3 and 4 in table 8 include control variables for transfer regulations, to see if 
tendencies to use analysis differ for this type of regulation. In general, the relationship between 
Use and Quality seems no different for transfer regulations that for non-transfer regulations. 
However, the transfer regulations that implement provisions of the American Recovery and 
Reinvestment Act appear to be marginally more likely to use the analysis. The Use score for 
these five regulations averages 7 points, compared to an average of 5 points for other transfer 
regulations in 2009. The difference in averages stems from relatively high Use scores for two 
Education Department regulations that provide grants to states for education reform: the School 
Improvement Grants (13 points) and the Race to the Top Fund (9 points). School Improvement 
Grants earned a relatively high Use score because the regulations focus the grants on education 
reforms that have research demonstrating their effectiveness, and because the regulation includes 


Using only Ihe four Analy sis crileria 5-8 as the independent variable produces rouglily ihe same results with a bit 
worse slalislical fit. 

' Results are in appendix 5. 
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provisions to gather data and evaluate the effectiveness of the reforms funded by the spending. 
The Race to the Top fund did not make much use of analysis to create the regulation, but it did 
establish goals and require states to submit data to evaluate the effectiveness of the reforms 
funded by the regulation. 

5.2 Ex-Ante Use vs. Retrospective Analysis 

The total Use score consists of scores for two types of criteria that might be affected 
differently by the quality of analysis. Criteria 9 and 10 assess the extent to which the agency used 
the analysis to make decisions in the proposed regulation. Criteria 1 1 and 12 assess the extent to 
which the agency provided for retrospective analysis in either the preamble to the regulation or 
the Regulatory Impact Analysis. To see whether Quality has different effects on these variables, 
table 9 replicates the regressions in table 8 using criteria 9-10 as a dependent variable and using 
criteria 1 1-12 as a dependent variable. 

The quality of analysis clearly has a positive, statistically significant correlation with both 
the use of analysis to craft the regulation and on provisions for retrospective analysis. The effect 
is about twice as large for the former as for the latter. 

The Year dummy variable, however, shows that Quality has a differential effect in 2008 
only for use of analysis to craft the regulation. Agencies were no more likely to make provisions 
for retrospective analysis in 2008 than in 2009. This is perhaps unsurprising, given that 
Executive Order 12866 and Circular A-4 place little emphasis on retrospective analysis. 

Finally, the Transfer dummy variable indicates that agencies were neither more nor less 
likely to use analysis in crafting transfer regulations or provide for retrospective analysis. The 
Recovery Act dummy shows that these regulations tend to have better retrospective analysis 
provisions — again largely because of the higher scores of the two education reform regulations. 

These regressions identify some significant correlations, but we are not sure if they imply 
causation. Perhaps decision makers choose to use analysis when they are confident it is higher 
quality. Or perhaps analysts prepare better analysis when they are confident the decision makers 
will use it. Similarly, the higher Use scores in 2008 might reflect a stronger commitment to using 
regulatory analysis in the Bush administration, but other hypotheses might also explain the 
difference. To the extent that regulations proposed in 2009 were already in process in 2008, 
perhaps the Bush administration simply pushed out the regulations that were better-supported by 
analysis in 2008 and left the rest for the Obama administration to deal with. Alternatively, the 
difference could just reflect the fact that 2009 was a transition year (perhaps because new 
members of an administration have to “learn” how to use economic analysis). Forthcoming data 
on the quality and use of regulatory analysis in 2010 may allow us to test these and other 
hypotheses. Systematic interviews of federal regulatory personnel, such as those conducted by 
Williams (2008), could provide additional (and perhaps even better) insights. 
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T«hle 8: Quality of Analysis vs. l<se of Analysts 


Dependent Vuriabh*: Uieuf Anahkb Sct>r« (Crftcria 9-12) 

EiipUnalitn 

Varislilw Lll 12J lil 111 


Oimlih 

0 30 

0 31 

0.27 

0.25 

(Cnlcnn 1-^J> 

Ki9«***| 

|7 28***| 

|3.9‘>***1 

(3 83***1 

Ycni 


1.34 

1 15 

1.33 

Oumim 


|2 3I***| 

|Ht5*I 

|2 I4**| 

Transfer 



-0.80 

-1 IV 

Rcgulalioii 



1-0.851 

I-I25I 

Rccovcr> ,\cl 




2 25 

Rcgulnlion 




|I 70*1 

Constiurt 

1 14 

33 

164 

I.N2 


112-11 

10.341 

i0 91| 

11021 

N 

87 

87 

87 

87 

Adjusted R' 

0.36 

0.39 

0.-39 

0.40 


Ordiiian least squares rcjp'e&sioits, i-stalislics lu paivnlhcscs 
Slnli&l»cal sigiuftcancc ***l|)crccnl **5 peroeni *10 percent 


Figure 3: L'se of Analysis Scores by Quintile 



Use score range 
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Table 9: Quality of Analysis vs. Separate Scores for Ex-Ante and Retrospective Analysis 



Dependent Variable: Ex Ante Use of Analysis (Criteria 9-10) 

Explanatorj' 

Variables 

(1) 

(2) 

(3) 


Qiialitv 

0.20 

0.20 

0.17 

0.17 

(Criteria 1-8) 

|6.05***| 

|6.30***| 

|.3.46***| 

|3.37***| 

Year 2008 


0.94 

0.83 

0.87 

Dmmnv 


[2.18**] 

[1.78*] 

[1.82*] 

Transfer 



-0.51 

-0.58 

Regulation 



[-0.72] 

[-0.80] 

Recoverv Act 




0.45 

Regulation 




10.451 

Constant 

0.34 

-0.22 

0.60 

0.64 


10.501 

|-().32| 

10.441 

10.471 

N 

87 

87 

87 

87 

Adjusted R" 

0.29 

0.32 

0.32 

0.31 



Dependent Variable: Provisions for Retrospective Analysis (Criteria 1 1-12) 

Explanatnry 

Variables 

(1) 

(2) 

(3) 

(4) 

Ouablv 

0.11 

0.11 

0.09 

0.08 

(Criteria 1-8) 

[3.98***] 

[4.04***] 

[2.19**] 

[2.00**] 

Year 2008 


0.39 

0.32 

0.47 

Dummy 


[1.06] 

[0.81] 

[1.29] 

Transfer 



-0.29 

-0.61 

Regulation 



1-0.491 

l-i.oii 

Recovery Act 




1.80 

Regulation 




|2. 15**1 

Constant 

0.79 

0.56 

1.04 

1.18 


11.391 

10.911 

|0.90| 

11.041 

N 

87 

87 

87 

87 

Adjusted R' 

0 15 

0.15 

0.14 

0.18 


Ordinaiy least squares regressions; t-statistics in parentheses. 
Statistical significance: ***1 percent **5 percent *10 percent 
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5.3 Use by Individual Agencies 

Is the reduction in Use scores widespread, or concentrated in a few agencies? Table 10 
sheds light on this question by calculating changes in average Use scores for individual agencies, 
including and excluding transfer regulations. 

Including all regulations, four agencies improved their average Use scores between 2008 
and 2009: Interior, Agriculture, Health and Human Services, and Veterans Affairs. Except for 
Agriculture, all of these improvements were less than one point. Seven agencies saw their 
average Use scores fall, and all of these reductions exceeded two points. Thus, improvements are 
small, and reductions are widespread. 

Some of these changes were driven by the increased proportion of transfer regulations in 
2009. Excluding transfer regulations, four agencies increased their Use scores: Interior, 
Agriculture, Health and Human Services, and Energy. Interior’s score increased by just 0.7 
point; all the others increased by at least two points. Four agencies saw their Use scores fall 
when transfer regulations are excluded; Homeland Security, Transportation, EPA, and Labor. 
Each of these four reductions was two points or greater. Excluding transfer regulations thus 
suggests that some agencies had noticeable improvements in their Use scores, while about the 
same number saw noticeable reductions. 

The changing mix of transfer vs. non-transfer agencies had a big effect on results for four 
agencies; Energy, Homeland Security, Transportation, and Health and Human Services. 
Excluding transfer regulations actually increases Energy’s Use score; with transfer regulations. 
Energy’s Use score falls. Excluding transfer regulations leads to a much bigger increase in 
Health and Human Services’ Use score; a 5.5 point increase instead of a 0.7 point increase. 
Finally, excluding transfer regulations cuts the reduction in Homeland Security’s and 
Transportation's Use scores by more than half 

The regression equations in tables 8 and 9 show that use of analysis to make decisions 
about regulations is lower in 2009, even after controlling for transfer regulations. Tabulations in 
table 10 suggest that the primary reason for the statistically significant decline in Use scores in 
2009 appears to be the reductions in Use scores at Transportation and EPA. Of all the agencies 
whose average Use scores fell. Transportation proposed two regulations in 2009 and EPA 
proposed nine. No other agency whose Use score for non-transfer regulations fell in 2009 
proposed more than one non-transfer regulation in 2009. 

In fairness, we should also note that the combined DOT/EPA CAFE/greenhouse gas 
emissions regulation earned the highest Use score in 2009: 15 points. In addition, the caveat we 
applied to table 7 applies to table 10 as well. Because the number of regulations is so small, it is 
hard to make reliable generalizations about particular agencies. For that, more years of data are 
needed. 
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Table 10: Use by Individual Agencies 


All Regulations 

2009 

Average 

Score 

# of 

Regulations 

2008 

Average 

Score 

# of 

Regulations 

2008-09 

Change 

Joint DOT/EPA 

15.0 

1 

NA 

0 

NA 

Interior 

9,0 

1 

8.3 

4 

+0.7 

USDA 

8,0 

1 

5.0 

1 

+3.0 

Energy 

7,4 

5 

10.0 

1 

-2.6 

EPA 

7,2 

9 

10.5 

2 

—3.3 

Education 

7,0 

5 

9.0 

2 

-2.0 

DHS 

6,5 

2 

12.0 

2 

-5.5 

HHS 

5,6 

12 

5.5 

11 

+0.1 

HUD 

5.0 

1 

10.0 

1 

-5.0 

DOT 

4.5 

3 

10.0 

6 

-5.5 

Labor 

4.0 

1 

8.7 

6 

-4.7 

Veterans 

3.0 

1 

2.0 

1 

+ 1.0 

Justice 


0 

11.7 

3 

NA 

Treasury 


0 

9.0 

1 

NA 

Fed Acquisition 


0 

4.0 

1 

NA 

SSA 


0 

3.0 

1 

NA 

State 


0 

2.0 

1 

NA 

Defense 


0 

1.0 

1 

NA 







Non-Transfer 

Regulations 

2009 

Score 

#of 

Regulations 

2008 

Score 

#of 

Regulations 

2008-09 

Change 

Joint DOT/EPA 

15.0 

1 

NA 

0 

NA 

Energy 

12.0 

3 

10.0 

1 

+2.0 

DHS 

10.0 

1 

12.0 

1 

-2.0 

Interior 

9.0 

1 

8.3 

4 

+0.7 

DOT 

8.5 

2 

10.0 

6 

-2.5 

USDA 

8.0 

1 

5.0 

1 

+3.0 

EPA 

7,2 

9 

10.5 

2 

—3.3 

HHS 

7,0 

1 

2.0 

2 

+5.0 

Labor 

4,0 

1 

8.7 

6 

-4.7 

HUD 


0 

10.0 

1 

NA 

Justice 


0 

11.7 

3 

NA 

Treasury 


0 

9.0 

1 

NA 

Federal Acquisition 


0 

4.0 

1 

NA 


Maximum possible Use score = 20. 
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6. Conclusions 

This study expands on existing research by applying a consistent set of standards to 
assess the quality and use of regulatory analysis for all economically significant regulations 
proposed in two different years. We find that the average quality of analysis is not high. The 
quality and use of regulatory analysis is especially poor for transfer regulations that define how 
the federal government will spend or collect money. But Regulatory Impact Analyses and 
Federal Register preambles present many examples of best practices that could improve the 
quality and use of analysis significantly if they were diffused more widely. 

Our comparison of regulations in 2008 and 2009 generates several insights relevant to 
contemporary regulatory policy discussions. We find very little evidence that the quality of 
regulatory analysis changed between 2008 and 2009. The most significant improvement occurred 
in accessibility of regulatory analyses on the Internet. While this is a welcome improvement that 
is consistent with the Obama administration's focus on government transparency, improvements 
on a few other criteria were generally small and, at best, usually improved average scores from 
poor in 2008 to middling in 2009. In addition, we find substantial evidence that agencies were 
less likely to use the analysis to make decisions about proposed regulations in 2009 than in 2008. 

This research also raises numerous questions that deserve further inquiry. We have not, 
by and large, identified why the quality and use of regulatory analysis exhibits the patterns 
revealed in this paper. For example, it is not obvious why some non-transfer regulations receive 
better analysis than others. Subject matter, deadlines, differing statutory mandates, explicit 
policy preferences, or department-specific factors may be part of the explanation. 

It is also not clear why the quality of regulatory analysis changed very little between 
2008 and 2009. Does this mean career staffers at agencies and/or OIRA consciously promote 
continuity between administrations? Another factor that may have played a role is that it is likely 
that the Bush administration focused greater effort on improving the quality of its “midnighf ’ 
final regulations in 2008 relative to its proposed regulations, while the Obama administration is 
likely to have placed a greater focus on its own newly proposed regulations. This would suggest 
that the quality of analysis for proposed rules should have improved in 2009 — unless most of the 
regulations proposed in 2009 were already in the pipeline in 2008. Research on what happened to 
the quality and use of analysis for final rules might shed further light on this issue. 

Our data also indicate a statistically significant reduction in OIRA review time for non- 
transfer regulations in 2009 (from 66 to 40 days), but not for transfer regulations, which 
averaged about 35 days in both years. McLaughlin (2010) finds that midnight regulations receive 
shorter review times at OIRA. Whether OIRA review time impacts quality and use is an area 
ripe for further research. 

Finally, we do not know why the use of regulatory analysis to make regulatory decisions 
declined in 2009. Indeed, we are not even sure if good analysis leads to use in decisions, or if 
decision makers’ openness to analysis promotes good analysis, or if some third set of factors 
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causes both of these. Creating consistent data on the quality and use of regulatory analysis is the 
first step toward answering these questions. 
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Appendix 1 

Major Factors Considered When Evaluating Each Criterion 

Note; Regardless of how they are worded, all questions involve qualitative analysis of how well 
the RIA and the l^’ederat Register notice address the issue, rather than ''yes/no” answers. 

Openness 

1. How easily were the RIA, the proposed rule, and any supplementary materials found online? 

How easily can the proposed oile and RTA be found on the agency's website? 

How easily can the proposed oile and RIA be found on Regulations.gov? 

Can the proposed rule and RIA be found without contacting the agency for assistance? 

2. How verifiable are the data used in the analysis? 

Is there evidence that the analysis used data? 

Does the analysis provide sufficient information for the reader to verify the data? 

How much of the data are sourced? 

Does the analysis provide direct access to the data via links, URLs, or provision of data in 
appendices? 

If data are confidential, how well does the analysis assure the reader that the data are valid? 

3. How verifiable are the models and assumptions used in the analysis? 

Are models and assumptions stated clearly? 

How well does the analysis justify any models or assumptions used? 

How easily can the reader verify the accuracy of models and assumptions? 

Does the analysis provide citations to sources that justify the models or assumptions? 

Does the analysis demonstrate that its models and assumptions are widely accepted by relevant 
experts? 

How reliable are the sources? Are the sources peer-reviewed? 

4. Was the agency’s analysis comprehensible to an informed layperson? 

How well can a non-specialist reader understand the results or conclusions? 

How well can a non-specialist reader understand how the analysis reached the results? 

How well can a specialist reader understand how the analysis reached the results? 

Are the RIA and relevant portions of the Rederal Register notice written in “plain English”? 
(Light on technical jargon and acronyms, well-organized, grammatically correct, direct language 
used.) 
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Analysis 

For each Analysis criterion, the lettered snh-qnestions each receive a score of 0 5, and these are 
averaged and rounded to produce the score on the criterion. 

5. How well does the analysis identify the desired outcomes and demonstrate that the regulation 
will achieve them? 

A. How well does the analysis clearly identify ultimate outcomes that affect citizens’ quality 
of life? 

B. How well does the analysis identify how these outcomes are to be measured? 

C. Does the analysis provide a coherent and testable theory showing how the regulation will 
produce the desired outcomes? 

D. Does the analysis present credible empirical support for the theory 

E. Does the analysis adequately assess uncertainty about the outcomes? 

6. How well does the analysis identify and demonstrate the existence of a market failure or other 
systemic problem the regulation is supposed to solve? 

A. Does the analysis identify a market failure or other systemic problem? 

B. Does the analysis outline a coherent and testable theory that explains why the problem 
(associated with the outcome above) is systemic rather than anecdotal? 

C. Does the analysis present credible empirical support for the theory? 

D. Does the analysis adequately assess uncertainty about the existence and size of the 
problem? 

7. How well does the analysis assess the effectiveness of alternative approaches? 

A. Does the analysis enumerate other alternatives to address the problem? 

B. Ts the range of alternatives considered narrow or broad? 

C. Does the analysis evaluate how alternative approaches would affect the amount of the 
outcome achieved? 

D. Does the analysis adequately address the baseline — what the state of the world is likely to 
be in the absence of further federal action? 

8. How well does the analysis assess costs and benefits? 

A. Does the analysis identify and quantify incremental costs of all alternatives considered? 

B. Does the analysis identify all expenditures likely to arise as a result of the regulation? 

C. Does the analysis identify how the regulation would likely affect the prices of goods and 
services? 

D. Does the analysis examine costs that stem from changes in human behavior as consumers 
and producers respond to the regulation? 

£. Does the analysis adequately address uncertainty about costs? 

F. Does the analysis identify the approach that maximizes net benefits? 



61 


30 


G. Does the analysis identify the cost-effectiveness of each alternative considered? 

H. Does the analysis identify all parties who would bear costs and assess the incidence of 
costs? 

I. Does the analysis identify all parties who would receive benefits and assess the incidence 
of benefits? 

Use 

9. Does the proposed rule or the RIA present evidence that the agency used the Regulatory 
Impact Analysis? 

Does the proposed rule or the RIA assert that the analysis of outcomes, benefits, the systemic 
problem, alternatives, or costs affected any decisions? 

How many aspects of the proposed rule did the analysis affect? 

How significant are the decisions the analysis affected? 

10. Did the agency maximize net benefits or explain why it chose another option? 

Did the analysis calculate net benefits of one or more options so that they could be compared? 

Did the analysis calculate net benefits of all options considered? 

Did the agency either choose the option that maximized net benefits or explain why it chose 
another option? 

How broad a range of alternatives did the agency consider? 

11. Does the proposed rule establish measures and goals that can be used to track the regulation’s 
results in the future? 

Does the RIA or Federal Register notice contain analysis or results that could be used to establish 
goals and measures to assess the results of the regulation in the future? 

In the RIA or the Federal Register notice, does the agency commit to performing some type of 
retrospective analysis of the regulation’s effects? 

Does the agency explicitly articulate goals for at major outcomes the rule is supposed to affect? 
Does the agency establish measures for major outcomes the rule is supposed to affect? 

Does the agency set targets for measures of major outcomes the rule is supposed to affect? 

12. Did the agency indicate what data it will use to assess the regulation’s performance in the 
future and establish provisions for doing so? 

Does the RIA or Federal Register notice demonstrate that the agency has access to data that could 
be used to assess some aspects of the regulation’s performance in the future'^’ 

Would comparing actual outcomes to outcomes predicted in the analysis generate a reasonably 
complete understanding of the regulation’s effects? 

Does the agency suggest it will evaluate future effects of the regulation using data it has access to 
or commits to gathering? 
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Does the agency explicitly enumerate data it will use to evaluate major outcomes the regulation is 
supposed to accomplish in the future‘s 

Does the analysis demonstrate that the agency understands how to control for other factors that 
may affect outcomes in the future? 
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Appendix 2: Crosswalk of 2010 OMB Regulatory Impact Analysis Checklist with Mercatus 
Regulatory Report Card evaluation criteria 


OMB Checklist 

Mercatus Evaluation Criteria 

Does the RIA include a reasonably detailed 
description of the need for the regulatory 
action? 

Criterion 6; How well does the analysis 
demonstrate the existence of a market failure or 
other systemic problem the regulation is 
supposed to solve? 

Does the RIA include an explanation of how the 
regulatory action will meet that need? 

Criterion 5; How well does the analysis identify 
the desired outcomes and demonstrate that the 
regulation will achieve them? 

Does the RIA use an appropriate baseline (i.e., 
best assessment of how the world would look in 
the absence of the proposed action)? 

Criterion 7, question D; Does the analysis 
adequately assess the baseline — what the state 
of the world is likely to be in the absence of 
further federal action? 

Is the infonnation in the RIA based on the best 
reasonably obtainable scientific, technical, and 
economic information and is it presented in an 
accurate, clear, complete, and unbiased manner? 

I Criterion 2; How verifiable are the data used in 
' the analysis? 

Criterion 3: How verifiable are the models or 
assumptions used in the analysis? 

Criterion 4; Was the analysis comprehensible to 
an informed layperson? 

Criterion 3 includes an assessment of whether 
the models and assumptions are based on peer- 
reviewed or othenvise reliable publications. 
However, the Mercams evaluation does not 
assess the quality of the underlying science. 

Are the data, sources, and methods used in the 
RIA provided to the public on the Internet so 
that a qualified person can reproduce the 
analysis? 

Criterion 1 takes the first step by assessing how 
easily the RIA itself can be found on the 

Internet. 

Criteria 3 and 4 include an assessment of how 
easily the reader could find the underlying data, 
sources, and methods from information or links 
provided in the RIA or the Federal Register 
notice. 

To the extent feasible, does the RIA quantify 
and monetize the anticipated benefits from the 
regulatory action? 

■ Criterion 5, question 2: How well does the 
analysis identify how the outcomes are to be 
measured? 
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To the extent feasible, does the RTA quantify 
and monetize the anticipated costs? 

Multiple questions under criterion 8 (Benefits 
' and Costs) assess how well the analysis 
identifies, quantifies, and monetizes costs. 

Does the RIA explain and support a reasoned 
determination that the benefits of the intended 
regulation justify its costs (recognizing that 
some benefits and costs are difficult to 
quantify)? 

Criterion 8, question F: Does the analysis 
identify the approach that maximizes net 
benefits? 

■ Criterion 8, question G: Does the analysis 
' identify the cost-effectiveness of each 
alternative considered? 

Does the RIA assess the potentially effective 
and reasonably feasible alternatives? 

Criterion 7: How well does the analysis assess 
the effectiveness of alternative approaches? 

Does the preferred option have the highest net 
benefits (including potential economic, public 
health and safety, and other advantages; 
distributive impacts; and equity), unless a 
statute requires a different approach? 

Criterion 10: Did the agency maximize net 
benefits or explain why it chose another option? 

Does the RIA include an explanation of why the 
planned regulatory action is preferable to the 
identified potential alternatives? 

Criterion 9: Does the proposed rule or RIA 
present evidence that the agency used the 
Regulatory Impact Analysis? 

Criterion 10; Did the agency maximize net 
benefits or explain whv it chose another option? 

Does the RIA use appropriate discount rates for 
the benefits and costs that are expected to occur 
in the future? 

Considered under criterion 5, question 2; How 

1 well does the analysis identify how the 
outcomes are to be measured?, as well as 
several questions about measurement and 

1 comparison of benefits and costs under criterion 
8 (Benefits and Costs). 

Does the RIA include, if and where relevant, an 
appropriate uncertainty analysis? 

Criterion 5, question E: Does the analysis 
adequately assess uncertainty about the 
. outcomes? 

Criterion 6, question D; Does the analysis 
adequately assess uncertainty about the 
existence and size of the problem? 

. Criterion 8, question E: Does the analysis 
. adequately address uncertainty about costs? 
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Does the RTA include, if and where relevant, a 
separate description of the distributive impacts 
and equity (including transfer payments and 
effects on disadvantages or vulnerable 
populations)? 

Criterion 8, question H: Does the analysis 
identify all parties who would bear costs and 
assess the incidence of costs? 

Criterion 8, question I: Does the analysis 
identify all parties who would receive benefits 
and assess the incidence of benefits? 

Does the analysis include a clear, plain- 
language executive summary, including an 
accounting statement that summarizes the 
benefit and cost estimates for the regulatory 
action under consideration, including the 
qualitative and non-monetized benefits and 
costs? 

Criterion 4: Was the analysis comprehensible to 
' an informed layperson? 

Does the analysis include a clear and 
transparent table presenting (to the extent 
feasible) anticipated benefits and costs 
(qualitative and quantitative)? 

Criterion 4; Was the analysis comprehensible to 
an informed layperson? 

Goals and measures lo assess resulls of the 
regulation in the future - No conterii. 

Criterion 1 1 : Does the proposed rule establish 
measures and goals that can be used to track the 
regulation's results in the future? 

Provisions for gathering data to assess results 
of the regulation in the fiiture - No content. 

Criterion 12; Did the agency indicate what data 
it will use to assess the regulation's 
performance in the future and establish 
provisions for doing so? 
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Appendix 3: Summary Statistics on All Criteria and Sub-Questions 
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Appendix 4: Average changes without separating transfer and non-transfer regulations 

The table below shows the change in average scores on individual criteria and on sub-questions 
for the Analysis criteria. We only report average scores whose differences are statistically 
significant at the 85 percent level or higher. Even for individual criteria or questions, there is 
very little evidence that average scores changed much between 2008 and 2009. As noted in the 
text, some of the changes identified below are driven by the increased proportion of transfer 
regulations in 2009. 

Score Changes on Individual Criteria and Questions 



2008 

(n=45) 

2009 

(n=42) 

Change 

T-stat. 

Openness 





Criterion 1 - Accessibility 

3.53 

4.05 

0.51 

2.10** 

Analysis 





Question 6B - Coherent Theory of Systemic Problem 

2.00 

1.50 

-0.50 

1.60 

Question 6C - Empirical Evidence of Svstemic Problem 

1.71 

1.21 

-0.50 

1.62 

Question 8C - Effects on Prices of Goods and Services 

1.38 

2.07 

0.69 

2.13** 

Question 8F - Identifies approach that maximizes net 
benefits 

1.91 

1.33 

-0.58 

1.62 

Use 





Criterion 10 - Decision Cognizant of Net Benefits 

2.20 

1.62 

-0.58 

1.80* 


Statistical significance: *90 percent **95 percent 

Maximum possible score on any criterion or question = 5 points. 


The increase on criterion 1 (Accessibility) indicates that agency regulatory analyses were 
somewhat easier to find online in 2009 than in 2008. This reflects the fact that regulatory 
analyses were easier to find on agency websites and Federal Register preambles provided clearer 
information about how to obtain a copy of the Regulatory Impact Analysis. Some of the 
improvement may also stem from the redesign of the regulations.gov web site, which may have 
made regulations and accompanying analysis easier to find. 

The lower average scores on questions 6B (Coherent Theory of Systemic Problem) and 
6C (Empirical Evidence of Systemic Problem) suggest that agencies may be somewhat less 
likely to demonstrate that proposed regulations actually address a market failure, government 
failure, or other systemic problem in 2009. Average scores were already quite low in 2008; this 
weakness may have gotten even weaker in 2009. 

The higher average score on criterion 8C (Effects on Prices of Goods and Services) 
indicates that agencies were more likely in 2009 to discuss the effects of regulatory costs on the 
prices of goods and services. This is something that agencies usually do either reasonably well or 
pretty poorly; there are few mid-range scores. The increase from 1 .38 to 2.07 implies that this 
improvement occurred only for a few regulations, or that agencies provided just a bit more 
discussion or evidence in place of unsupported assertions. 
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The lower scores on question 8F (Identifies Alternative that Maximizes Net Benefits) and 
criterion 10 (Decision Cognizant of Net Benefits) suggest that regulatory analyses in 2009 were 
less likely to assess the net benefits of alternatives, and decision makers were less likely to 
consider net benefits when choosing among alternatives. Agencies usually do these things either 
reasonably well or not at all, so this shift suggests that fewer regulations in 2009 identified or 
considered net benefits of alternatives. 
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Appendix 5: Use vs. Quality Employing Quality x Year Interaction Variable 



Dependent Variable: Use of Analysis Score (Criteria 9-12) 

Explsmatorj' 

Variables 

(1) 

(2) 

(3) 

(4) 

Quality' 

().3() 

0.28 

0.23 

0.22 

(Crileria 1-8) 

[6.98***] 

[6.26***] 

[3.67***] 

[3.41***] 

Year 2008 Duimny 


0.06 

0.05 

0.06 

X Quality 


|2.2I***| 

1 1.79*1 

|l. 98**1 

Transfer 



-0.88 

-1.28 

Regulation 



[-0.95] 

[-1.34] 

Rccovcrv' Act 




2.07 

Regulation 




11.571 

Constant 

1.14 

1.06 

1.64 

2.70 


11.241 

11.181 

10.911 

11.631 

N 

87 

87 

87 

87 

Adjusted R" 

0.36 

0.39 

0.38 

0.40 


Ordiiian- least squares regressions; T-stotistics in parentheses. 
Statistical significance: ***1 percent **5 percent *10 percent 
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Mr. Coble. Dr. Williams. 

TESTIMONY OF RICHARD A. WILLIAMS, DIRECTOR OF POLICY 
RESEARCH, MERCATUS CENTER, GEORGE MASON UNIVERSITY 

Mr. Williams. Chairman Coble, Ranking Member Cohen, and 
Members of the Subcommittee, thank you for the invitation to tes- 
tify today. My name is Richard Williams. I have been involved in 
rulemaking and the regulatory process for over 30 years, first as 
a regulatory analyst, and then as a senior manager for the Food 
and Drug Administration, and those are issues I continue to care 
deeply about. I also worked briefly in the Office of Information and 
Regulatory Affairs. Today, I serve as director of policy studies at 
the Mercatus Center at George Mason University. 

From my experience in research, I believe the regulatory process 
we have today is not what was originally intended 60 years ago 
when we passed the Administrative Procedures Act, which was to 
create a rational, transparent, and inclusive process. When the 
APA was passed, then-Senator Pat McCarran, who was the Chair- 
man of the Committee on the Judiciary, called it a widely heralded 
advance in democratic government. 

But one of the biggest problems with our regulatory process is 
that decisions are made first without any analysis or whether or 
not we know regulation is needed. And more importantly, without 
stakeholders being involved in that decision. By the time a pro- 
posal is generated, most of the significant issues have been decided 
and rules are steamrolled through to the final rule. The reason reg- 
ulatory agencies decide early is that their incentives are to crank 
out new regulations, whether needed or not. 

After all, passing regulations is the business of regulatory agen- 
cies, and their success is measured and rewarded based on this ac- 
tivity. The problem with deciding early is that we are regulating 
in the dark, that is without sufficient knowledge of whether a regu- 
lation is needed or will work. This isn’t just FDA. 

In a paper I did interviewing senior economists in many regu- 
latory agencies, I discovered that virtually all agencies make deci- 
sions early, and anyone inside or outside the agency who tries to 
suggest that regulation is not necessary generally finds that view 
unwelcomed. 

Analyzing that problem is part of what economists do in regu- 
latory impact analyses; but if your goal is to regulate as opposed 
to solving a problem, there is no reason to wait for analysis. Per- 
haps that is one reason why my colleague. Dr. Ellig, shows that de- 
fining the problem is one of the agency’s biggest issues: if you are 
vague about the problem you are trying to define, then no one can 
accuse you of failing to solve a problem. 

Another problem with deciding early is that stakeholders end up 
commenting on decisions, not problems. Now some stakeholders do 
get their voices heard in agencies prior to decisions being made, 
but these are often firms or activists who want a particular regula- 
tion to serve their own ends. As this also advances what the agency 
is trying to do, those are welcomed voices. But those that are con- 
fined to the comments after proposals have been issued will find 
any objections they raise to regulating will receive a short and deci- 
sive dismissal. And it normally goes something like: the agency dis- 
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agrees or you didn’t provide enough information to convince the 
agency. 

The result of all this activity is that we end up regulating far too 
often when it is not effective or not needed. That is why we now 
have 226 volumes of regulations, taking up 163,000 pages of rules, 
and we still continue to add 4,000 new regulations each year with 
this broken process. 

Certainly one tragic outcome of this, beyond our effect on com- 
petitiveness, is we can’t focus our resources on regulations that are 
truly needed and effective. We can fix this by changing the incen- 
tives that agencies face. However, we can no longer rely on execu- 
tive orders as every President since Richard Nixon has tried to do. 

Only Congress can fix this. One way they can do it is by statu- 
torily insisting that agencies start with a step to determine if a reg- 
ulation is necessary. This would be something like a preproposal 
publication that the agency would investigate and contain elements 
like a clear definition of the problem that the agency seeks to solve 
and the evidence that it relied on to define the problem, an expla- 
nation of and evidence for why the problem warrants Federal inter- 
vention, an exploration of a range of options that the agency be- 
lieves might solve the problem, and a preliminary estimate of bene- 
fits and costs of each option. 

All of this would be published to provide stakeholders and the 
public an opportunity to evaluate the agency’s data and research, 
and contribute additional information. As OIRA Director Sunstein 
says, they can take advantage of the fact that knowledge is widely 
dispersed in society, and public officials can benefit from access to 
that dispersed knowledge. In commenting on a preliminary anal- 
ysis like that, stakeholders and the public are much more likely to 
comment broadly from their collective wisdom and expertise on 
both what the actual problem is and whether or not it needs a reg- 
ulatory solution. Use of this knowledge will help us solve our prob- 
lems much more effectively than our current process. While much 
more needs to be done to address institutional barriers to problem- 
solving, an essential first step will be in the establishment of some 
kind of an evidence-driven and inclusive process for defining the 
problem and the potential options to address it. 

Thank you. 

Mr. Coble. Thank you. Dr. Williams. 

[The prepared statement of Mr. Williams follows:] 
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Mr Chaimion and Members of the Committee: 

Tli.'uik you for the invitation to testily today on die 'Raising the Agencies Grades — protecting the 
economy, assuring regulatory quality and tmprov ing assessments of regulatory needs - I am an 
economist and the director of policy studies at the Mercatus Center, a $0l(c) (3) msearch. educational, 
and outreach oiganization affiliated with George Mason Unuersity For over three decades. I have been 
involved in miemaking and regulatory analy sis. first as a regulatory analy st tlicn senior manager at the 
Center for Food Safety and Applied Nutrition (CFSAN) at the Food and Drug Administmlion (FDA). I 
also worked for a short time in the Office of Informabon and Regulatory Affairs (OIRA) in die Office of 
Management and Budget (OMB) reviewing rules from other agencies 

The United States has a large regulatory system that is producing regulations at a fantastic rate. Each 
year, the Federal Register publishes nearly ilO.OOO pages, the equivalent of 400 novels, that announce new 
regulations Tlic result of this publication is the production of over 4.0(K) regulations per y car 

The regulatory system is large, but regulations are largely produced in the dark In many eases, regulators 
don't even know if there is a problem that needs to be solved, much less the best w ay to solve a potential 
problem. They don't know if the public thinks there is a problem They don't know if the public has 
belter solutions for any potential problem 

They don't know, because it's not ui die agencies' interests to know If an agency were to determine 
through regulatory analy sis that there rs no problem or that a method other than federal regubtion will 
solve the problem, what’s its reward'! Agencies arc not rewarded, cither with budgets or political support, 
for not passing regulations They are rewarded for producing diem Further, they are rarely exconated for 
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producing unnecessar}- or ineffective rules, but they are frequently talcen to task when the slightest 
problem manifests itself in any area under their jurisdiction for failure to regulate or enforce. 

Because of the incentives they tace, agencies make decisions to regulate before any evidence that might 
suggest regulations are not needed. They do so purposely with little — if any — input from stakeholders or 
internal analysis. This means that much of tlie regulator^' process mandated b\' tlie Administrative 
Procedures Act is a farce in tenns of the intent of that Act. According to Senator Pat McCarran, then 
Chairman of the Committee on the Judiciar\- in 1 946, the APA was intended to be a "widely heralded 
advance in democratic government.''’ 

Our institutions, including our Executive Orders and all of our existing regulator)- procedural laws, have 
brought us to this point, and only a change in our regulator)' institutions will change the outcomes. Our 
goal ought to be fewer, more targeted, and more effective regulations that solve real problems. 

WHAT ACTUALLY HAPPENS 

At FDA, the informal ailemaking process begins with new legislation, a petition, or an internal push to 
create a new regulation, Following that, a senior program manager typically makes a preliminary 
decision about the regulation and then holds a meeting to let every-one know what that decision is. 
Discussion of whetlier tliere is a problem and whether federal regulation is the best way to solve that 
problem is “off tlie table." That is, dicrc is no discussion of whether or not a regulation is required. Tlicrc 
is also no discussion as to whether there is a failure of the market or some other reason for regulator)- 
intervention; whether the market will solve the problem in the near friture without intervention (baseline 
analysis); or if there is a need for federal, as opposed to some other level of government, inter\'ention. 
Tliese are issues that are lypicalh' covered in the regulator)- impact analysis. But that analysis is generally 
begun after die decision on how to regulate has been announced. That is a key part of die problem: the 
regulatory analysis analyzes a decision, not a problem. 

Oue ke)' part of the analysis that would be extremely helpful if done before a decision is a baseline 
analysis. Usually by the time the govemmeut has discovered a problem, the market is also moving to 
solve the problem. In the baseline analysis, economists will not just describe the state of the world as it is 
today, but what is likely to happen given the incentives created by the market.” Baseline analysis will 
describe how both knowledge and practices are likely to change for both producers and consumers in the 
near future in the absence of a regulation. So. for e.xample, if diere is an outbreak of food borne disease 
tied to a particular producer of a canned vegetable, then the producer’s sales will drop, its stock price will 
drop, and it will face court challenges. Other producers of that canned vegetable wall examine their own 
pltuits to see if tliey if they can avoid similar problems because it is in their best interest to do so. If, for 
some reason, tliese incentives were missing, and there were repeated problems overtime (i.e., systemic 
problems) with known solutions, then a regulation might help. A baseline analysis can help to sort out 
which problems are systemic and wEich ones are one-time problems that markets will solve. 


^ Robert Boyd, “A Hearing Examiner Comments on the APA and the Rule Making or Adjudication Controversy, 
“William and Mary Law Review 2 (1960). 

http://scholarship.law.wm.edu/cgi/viewcontent.cgi?article=2796&context=wmlr&sei- 

redir=1#search="legislative+history+testimony+administrative+procedures+act+1946. 

^ Market failures are often market opportunities for entrepreneurs who will seek to profit by finding and marketing 
remedies for those problems. 
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As the decision moves up the chain, more senior managers will typically ask questions about any 
potential political opposition they are likely to encounter. Discussion of regulatoiy- options, which are 
often only about minor adjustments, occurs much later in the process, but it is rare for any major change 
from die early decision. 

OPACITY NOT TRANSPARENCY 

Federal agencies are contained in tlieir own little microcosm. Tliey need the vast supply of knowledge 
that conies from stakeholders. As the Administrator of OIRA Cass Sunstein says. ''Government should 
actively solicit expertise from outside Washington so that it makes policies widi the benefit of die best 
information.’' ■ But agencies do not seek stakeholder input, particularly stakeholder input that might 
suggest that no regulation is needed when ‘'making policies.” This is not surprising. There is no incentive 
for them to do so. 

The incentives that exist in the current system to take contrary- stakeholder opinions into account are, at 
best, weak, and they don’t even apply mitil late in the rulemaking process. Most regulations today are 
passed by what is known as infomial mlemaking. also known as notice and comment. Federal law-s that 
gave us notice and comment have also made federal documents and meetings available to all 
stakeholders. But that access and transparency do not equate to having equal ability' to influence rules. In 
informal rulemaking, agencies are required to respond to comments made to proposals. But too often 
responding to contrary' arguments means dismissing them swiftly and easily. Tliis doesn’t mean that no 
stakeholders have influence over the early decisions. Generally, those that have petitioned for and favor 
regulations are heard from early in die process to help shape the initial decisions. 

In a rule on seafood processing that I worked on at FDA, the vast majority of commenters to the proposal 
disagreed with FDA’s approach. But the summary in the final rule said tliat ‘‘some people agree and 
some people disagree,” A common dismissal that rule writers use is the commenter did not provide 
"sufficient evidence to alter the agencies conclusion.” A simpler one is just that ‘the agency disagrees 
ivith the coiiunenter.” Having leapt the low hurdle, the agency proceeds to steamroll the decision made 
early on in tlie process. 

A REAL SYSTEMIC PROBLEM 

For virtually my entire career at FDA, 1 worked to improve the Center for Food Safety and Applied 
Nutrition’s (CFSAN) regulatory' analysis to produce better, more informed rules. At one point, 1 designed 
a Standard Operating Procedure (SOP) for use in the development of new regulations. In its simplest 
form, tlic SOP sought to ensure that the agency would define die problem, identify’ tlic options for solving 
it, and conduct all the relevant analyses (c.g.. scientific, economic, and legal) before initiating new 
mlemaking. 

The SOP was included in a lengthy code of practices that was supposed to determine how regulations 
•would be developed in CFSAN. However, once it was developed, agency program managers essentially 
ignored it by continuing to make decisions first and then starting the regulatory process. By making 


^ Cass Sunstein, "The Power of Open Government" (speech, The Brookings Institution, Washington, DC, March 1 0, 
2010) http://www.whitehouse.gov/omb/oira_speech_03102010/. 
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decisions first, the more basic questions about defining the problem, whether or not we actually needed a 
regulation, and a discussion of the options were completely precluded. 

Curious as to whether other economists working in the health, safety and environmental areas had 
agencies similar to mine, T inter\'iewed senior economists in all of the health, safety, environmental, and 
security’ agencies for information on how tliey conducted their respective regulatory’ processes and how 
policy mailers used tlicir analysis.'^ The findings indicated that policy makers use regulatory analysis as a 
means to justify a prc-dctcrmincd course of action, not a tool for problem-solving. 

1 ) Agency policy’ makers commonly make decisions well prior to the completion of tlie regulatory 
analysis. 

2) Agency managers often discourage comprehensive, quality economic analy sis. As one economist 
put it, “U think it would have an immense difference if we were allowed to do good regulatory’ 
analysis.”^ 

3) Some economists noted that their agencies often did not define the problem. 

4) Another economist noted that agency policy makers ''foreclose any discussion of meaningflil 
options [for solving a problem] at a pretty early stage. Another said, ‘'We do what we always 
do: just trotting out tlic same old thing. That's why wc don’t conic up with better regulations; vve 
just come up ivitli the same regulations in diftcrent areas.’" Anotlicr described tlic atmosphere for 
tliosc who try' and talk about regulatory options as ‘'oppressive." 

5) Because they decide the course early in the process, agency policy malccrs put a lot of pressure on 
economists to come up with the ‘'right’ answer, one that justifies their premature decisions, Tlie 
pressure works because the economists are organizationally under the people making the 
decisions, Tliis is an enomious problem for federal regulatory economists. Witliin agency 
bureaucracies, pointing out potential flaws in one's superior's decision rarely helps tlic course of 
one’s career. 

INCENTIVES 

Tliese findings are not une.xpected. Scholarly research evaluates and aiiEilyzes tlie incentives that drive 
agencies to produce regulations. And, to the best of my knowledge, tlicrc arc no tlicorics that hypotlicsizc 
that there are incentives for agencies not to regulate. 

Some examples of the theories of regulation include public interest theories that generally point to market 
failures and assume that governments can correct market problems (ivithout any problems with the 


'' Richard A. Williams, “The Influence of Regulatory Economists in Federal Health and Safety Agencies” (working 
paper, Mercatus Center at George Mason University, Arlington, 2008) 
http://mercatus.org/sites/default/files/publication/WP0815_Regulatory%20Economists.pdf. 

^ Ibid. 

®lbid. 
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operation of government).^ The "capture theorN-” suggests that industries will '"capture” the agencies that 
regulate them, causing the agencies to produce regulations that benefit those industries. Industries are able 
do this because tlie benefits of the regulation go to concentrated industries that can launch an effective 
lobby ing campaign whereas the cost arc spread out to a broad, a diffuse group, typically consumers. Dr. 
Bruce Yandlc of Clcmson University takes this idea a step further widi his tlicoiy of "Bootleggers and 
Baptists.”^ This thcoiy suggests that industries will tacitly join with activists seeking to advance a cause 
and align their interests to create regulation. For example, environmental activists may find an 
unexpected regulatory ally in the fight for clean air in a firm that sells smokestack sembbers. 

Private-interest theories hypothesize that politicians and bureaucrats are self interested and pass laws and 
regulations to reward tliemselves. It is certainly true that both agencies and bureaucrats within agencies 
have their own goals. Bureaucrats desire "salary-, perquisites of tlic otficc, public reputation, pow cr, 
patronage, ease of managing the bureau, and ease of making changes” and all areas are ^'positive 
monotonic function of the total budget of the bureau As one writer puts it. "Despite the absence of any 
obvious increase in social product on its part, the bureaucracy shows an insatiable appetite for consuming 
an increasing share of resources over time.”’^ In fact, as another writer points out, "Budgets. . . are a 
recurring and pen-asive need: indeed, they arc often the ‘bottom line' in tlic matter of sur\ival.” 

In all cases, tliere is an absence of incentives to use rational analysis based on extenor incentives such as 
is found in tlic private sector. In private firms, there is a reason to control costs and target specific 
markets. Businesses have an instantaneous feedback from the market, tlic effect on profits of poor 
decisions. For example, if costs arc excessive, then revenue will not cover costs or generate a profit. 
Similarly, if marketing targets consumers or firms who are unlikely to buy the product, then costs wall 
again exceed the revenue generated from those efforts. If a finn does not make a profit or cover its costs, 
it goes out of business. 

Regulatory' agencies do not have a similar feedback mechanism to help distinguish vvell-infomied 
decision mailing from prc-dctcmiincd decision malcing. Without feedback, there isn't sufficient incentive 
to ensure that decisions arc made after stakeholder input and rigorous analysis of the need for and 
consequences of regulation. 

ALTERING INCENTIVES 

Changing die incentives so that agencies regulate only when diere is a systemic, significant problem that 
whll not solve itself and needs to be addressed at the federal level will require major structural changes in 
the institutions that govern and reward agencies. Oue thing that will make a diftereuce immediately is to 
decouple the agency's decision from both early analysis of and democratic input into a problem. That is. 


^ See, for example, Johan den Hertog," General Theories of Regulation," 224, 
http://www.scribd.com/doc/239491/General-Theories-af-Regulation. 

® Bruce Yandle, “Bootleggers and Baptists — The Education of a Regulatory Economist," Regulation ( May/June 
1983): 12-16. 

® William A. Niskannen, “Nonmarket Decision Making, the Peculiar Economics of Bureaucracy," The American 
Economic Review 5Q (1968): 293-4. The hypothesis is challenged in Ronald N. Johnson and G. Libecap, “Agency 
Growth, Salaries and the Protected Bureaucrat,” Economic Inquiry, 27 (1989). 

Daniel K. Benjamin, “A Users Guide to the Regulatory Bureaucracy,' in Regulation and the Reagan Era. eds. 
Roger E. Meines and Bruce Yandle (New York: Holmes and Meirs, 1989), 253. 

Paul J. Quirk, Industry Influence in Federal Regulatory Agencies (Princeton: Princeton University Press, 1981), 96. 
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initially, agencies should perform regulatoiy’ analysis and make that analysis available for public 
comment. There should be no discussion of the agency’s preferred solutions in this document. 

Specifically, this document should contain: 

1) A clear definition of the problem that the agency seeks to solve and the evidence it relied on to define 
the problem. An agency could not satisfy this requirement by simply stating that it is issuing a rule in 
order to satisfy an enabling statute. Instead it must specify' the contours of a significant, systemic 
problem and the evidence it used to identify those contours. If the problem involves risk, a risk 
assessment should be provided. 

2) An explanation of and evidence for why a federal solution is required for this problem including: 

i) Is tlic problem a failure of the market or other significant problem? 

ii) Is it a large enough problem to warrant a federal solution, particularly given other priorities? 

iii) Is it a problem better solved at the state or local level? 

iv) Will nomial market forces solve the problem within the relevant timeframe? Tlie answer to 
this question comes from the above-described baseline analysis. 

3) The possible ways to solve the problem. One answer may be to gatlicr more infonnation particularly 
if there are no solutions that are effective or cost-beneficiaJ . Options generated should be broad 
including, for example, providing or requiring information, providing guidance, establishing 
performance objectives, and technology-based standards (command and control). Alternatives can 
also include levels of stringenev . covering different sectors of the economy, different time frames for 
compliance, exemptions for certain sizes of firms, encouraging industry -established standards, or 
enhancing liability laws. 

4) A preliminary estimate of the benefits and costs of each of the options listed. 

All of this analysis should be of a regulatory' problem, not a regulatoiy decision. This pre-proposal 
document should mal^c all data and assumptions available to the public, and the agency should request 
public comments on all parts of the document. Tliis publication would serve several purposes. First, it 
would mal<.c the agency's analysis and data transparent to interested parties and tlic public. Second, it 
W'Ould provide interested parties and the public with an opportunity to evaluate the agency’s data and 
research and contribute data and information that the agency may not have identified. Third, it would 
reinforce the agency's focus on problem-solving rather tlian the narrower activity of promulgating 
regulation. 

CONCLUSION 

As an earlier Congressional testimony noted, "Often what occuk before a notice of proposed rulemaking 
has been published produces commitments that, in the words of President George H.W. Bush's General 
Counsel at the EPA, convert notice and comment rulemaking into a fomi of Kabuki theater — ‘a highly 
stylized process for displaying in a fonnal w'ay the essence of something which in real life takes place in 
other venues.’"^' 


Subcommittee on Courts, Commercial and Administrative Law Committee, The APA at 65 — Is Reform Needed to 
Create Jobs, Promote Economic Growth, and Reduce Costs? Hearing before the Committee on the Judiciary, United 
States House of Representatives, 112‘^ Cong., 1®‘sess. (February 28, 2011). 
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Regulation though isn't a theatrical performance. Regulator)^ agencies, by virtue of their extensive reach 
into eveiy’ aspect of American life, have become big business, and they are not well am. Decisions are 
made with minimal democratic input and analysis ofwhetherornot regulations are necessaiy-. Decisions 
arc made this way because the institutions that govern agencies rulcmalcing create incentives to do so. If 
we change the institutions, we will get fewer, more effective, and more targeted regulations that will help 
to solve our serious social problems. 
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Mr. Coble. We have been joined by the distinguished gentleman 
from Georgia, Mr. Johnson. Good to see you, Hank. 

Professor Glicksman, good to have you with us. 

TESTIMONY OF ROBERT L. GLICKSMAN, J.B. AND MAURICE C. 

SHAPIRO PROFESSOR OF ENVIRONMENTAL LAW, GEORGE 

WASHINGTON UNIVERSITY LAW SCHOOL 

Mr. Glicksman. Chairman Coble, Ranking Member Cohen, 
Members of the Subcommittee, thank you for inviting me to testify 
today. 

My name is Robert Glicksman. I am the J.B. and Maurice C. 
Shapiro Professor of Environmental Law at The George Wash- 
ington University Law School, although I am here today strictly in 
my personal capacity. 

The premise of the Mercatus Center’s regulatory report card 
project is that the Federal rulemaking process is flawed, and the 
best way to fix it is for agencies to engage in more rigorous regu- 
latory analysis to provide better justifications of the need for and 
content of. More regulatory analysis, the idea is, particularly at the 
initial stages of regulation, would help avoid unnecessary regula- 
tion. The report card also reflects the conviction that cost benefit 
analysis is essential for identifying counterproductive regulations. 

My first response is that the framing of the problem is not one 
I would agree with. And to begin with, cost-benefit analysis, in my 
view, is itself a flawed technique for distinguishing between useful 
and counterproductive regulations. More fundamentally, the prob- 
lems arising from the current regulatory process, for the most part, 
are not the result of regulations lacking justification or whose costs 
exceed their benefits. Instead, the primary problem is inadequate 
resources to allow agencies to fulfill their statutory responsibilities 
and fulfill their tasks of achieving public policy goals. 

In addition, the regulatory process already allows those affected 
by regulation to identify flaws in agency regulatory proposals, and 
affords both regulated entities and agencies opportunities to fix 
problems, such as overly costly or unfair regulation. 

I want to make five points. 

First, I think the presumption that we can get better regulation 
if we make cost-benefit analysis more rigorous is just wrong. Cost- 
benefit analysis is inescapably limited by the difficulty of predicting 
and quantifying regulatory costs and benefits. Quantifies cost-ben- 
efit analysis requires agencies to reduce regulatory benefits, such 
as lives saved, to a crude dollar figure, so that the monetized bene- 
fits of regulation can be measured against its monetized costs. 
Some agency estimates of monetized regulatory benefits are ab- 
surdly low, and I have given some examples in my statement. 

Beyond that, inconsistencies in how agencies monetize benefits 
cast doubt on the usefulness of the effort. Cost benefit figures, 
therefore, provide a misleading aura of precision and rationality. 
Monetization of benefits often depends on arbitrary assumptions 
that tend to undervalue social benefits of regulation that are hard 
to quantify. Efforts to reform the methodology for cost benefit anal- 
ysis will at best yield only marginal improvements in regulatory 
decisionmaking. And, therefore, cost-benefit calculations should be 
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used with caution and with an acknowledgment of their limita- 
tions. 

Second, the real problem to which Congress should be directing 
its attention is not insufficient agency focus on cost-benefit meth- 
odologies, but the destructive convergence of funding shortfalls, de- 
monizing political attacks and outmoded legal authority, all of 
which have set the stage for ineffective enforcement and unsuper- 
vised industry self-regulation. Examples, some of which were point- 
ed out by Representative Conyers and Representative Cohen al- 
ready today, include the Deepwater Horizon spill in the Gulf of 
Mexico; the mine disaster at the Upper Big Branch Mine, West Vir- 
ginia; a peanut products tank tainted by salmonella; glasses con- 
taminated by cadmium sold to children at fast food restaurants; 
Code Red smog days when parents are warned to keep their chil- 
dren indoors; and the recall of widely used pharmaceutical drugs 
found to create risks of heart failure. 

All of these instances reflect agencies unable to do their jobs in 
protecting the public interest. More analysis will not fix these 
flaws. If anything, more analysis will only make things worse by 
slowing agencies down without demonstrably improving the quality 
of their regulatory decisions. 

Proponents of cost benefit analysis remain focused on perfecting 
formulas, assumptions, models, and data sets. If we really want to 
fix the regulatory system, we should, instead, focus on finding ways 
to help agencies effectively and efficiently achieve their statutory 
missions of protecting and the environment. 

Measures that I think would move us in that direction include 
providing agencies with the resources they need and enhancing 
their legal authority in situations in which it has become out- 
moded. 

Third, current law provides ample opportunities to address un- 
certainty, unnecessary or ill-advised regulation without heaping on 
agencies already stretched to the limit more onerous analytical re- 
sponsibilities. The Administrative Procedure Act, in particular, pro- 
vides notice and comment process which affords regulated entities 
and others opportunity to provide input before a regulation goes 
into effect. 

Fourth, even if agencies get it wrong during initial rule promul- 
gation, the regulatory process allows those affected by regulation in 
unintended or counterproductive ways to seek relief from the agen- 
cy in the form of waivers and exceptions. 

Fifth, agencies can revise rules that don’t work out as intended, 
either because they turn out to be too weak or too strong. And, fi- 
nally, judicial review provides a check on unjustified regulation. 

I will just close by quoting from two former EPA administrators, 
Ruckelshaus and Whitman, who made this statement just last Fri- 
day in an op-ed in the Post: Our country today needs what it need- 
ed in 1970: a strong, confident, scientifically driven, transparent, 
fair, and responsible set of protective agencies, such as the EPA. 
Congress should help America achieve that. 

They also warned that those who do not support those goals 
should be aware that the American public will not long stand for 
an end to regulation that have produced their health and quality 
of life. 
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Thank you for the opportunity to talk to you today. 

Mr. Coble. Thank you, Professor. And to all witnesses, thank 
you for your testimony. 

[The prepared statement of Mr. Glicksman follows:] 


Statement of Robert L. Glicksman 

to the Honse Judiciary Committee’s Subcommittee ou Courts, Commercial and 
Administrative Law 

Raising the Agency’s Grades — Protecting the Economy, Assnring Regnlatory Qnality and 
Improving Assessments of Regulatory Need 

March 29, 2011 

My name is Robert L. Glicksman. I am the J.B. & Maurice C. Shapiro Professor of 
Environmental Law at The George Washington University Law School. 1 am also a member 
scholar at the Center for Progressive Reform. I graduated from the Cornell Law School and have 
practiced and taught environmental and administrative law for nearly 35 years. 

Improving Cost-Benefit Analysis Will Not Improve Regulatory Decision-Making 
The essential premise of the Mercatus Center’s Regulatory Report Card project is that the 
output of federal regulatory agencies is flawed and that the best way to address that problem is 
for agencies to engage in more rigorous regulatory analysis to provide better justifications of the 
need for and content of regulation. Implicitly, the premise seems to be that agencies are engaged 
in excessive regulation and that regulatory analysis, particularly at the initial stages of regulation, 
would cure that problem by demonstrating that regulation is unnecessary. The Report Card 
project also reflects the conviction that cost-benefit analysis is an essential analytical tool 
capable of identifying regulations whose costs to society exceed their benefits and that are 
therefore counterproductive. My first response to that framing of the problem is that cost-benefit 
analysis is itself a flawed technique for distinguishing between useful and counterproductive 
regulations. More fundamentally, while the current regulatory process is indeed flawed, the 
problems for the most part are not the result of agencies adopting regulations without 
justification or regulations whose social costs exceed their benefits. Instead, the primary 


2 



83 


problem is regulatory dysfunction resulting from providing agencies with inadequate resources 
to fulfill their statutory responsibilities, not giving agencies sufficient tools to address significant 
health , safety, and environmental risks, and burdening agencies with what are already excessive 
and unhelpful analytical obligations. Finally, the existing regulatory process already allows 
those affected by regulation to identify flaws in agency regulatory proposals and affords both 
regulated entities and agencies opportunities to fix problems such as overly costly or unfair 
regulation. 

Let me say at the outset that T’m delighted to see that the Report Card project that is the 
focus of this hearing acknowledges that there are in fact benefits to regulation. We’ve heard a 
full-throated assault on regulation over the last couple of months, in which a grossly inflated 
dollar figure for the supposed cost of regulation has been bandied about frequently. Those who 
have assailed supposedly excessive regulatory costs almost never mention the benefits of 
regulation, however, even though ample data indicate that these benefits vastly exceed the costs. 
Even using the deeply flawed methodology of cost-benefit analysis, which is slanted heavily 
against protective regulations, the benefits of regulation vastly exceed the costs. 

Putting aside the critics’ tendency to ignore or give short shrift to the benefit side of the 
equation, I think the presumption that we can get better regulation if we make cost-benefit 
analysis more rigorous is Just wrong. It sounds reasonable, but in the real world it will lead to 
longer delays in much-needed regulations, all in pursuit of some sort of mathematical ideal that 
is, in the end, illusory. 

As a decision-making methodology, cost-benefit analysis is inescapably limited by what 
the academic literature refers to as indeterminacy. By that, 1 mean that it produces cost and 
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benefit estimates that are so ambiguous and uncertain that they can tell us very little about the 
economic efficiency, “smartness,’' or quality of a regulation. 

One of the biggest sources of indeterminacy in cost-benefit analysis comes from its 
reliance on monetization, the process by which the cost-benefit analyst attempts to reduce any 
value — no matter how complex or how important — to a crude dollar figure, so that the 
monetized benefits of regulation can be measured against its monetized costs. On their face, 
some agency estimates of monetized regulatory benefits are absurdly low. For the 
Environmental Protection Agency’s (EPA) air pollution regulations, for example, the prevention 
of a non-fatal heart attack in a person 0-24 years old is worth only $84,000 and the prevention of 
an emergency room visit to treat an asthma attack is worth only $363.^ Protecting children’s 
developing brains against mercury pollution is worth only $8,800 per IQ point saved. ^ Until 
recently, agencies assigned a value of $0 to preventing catastrophic climate change, because this 
benefit was too hard to monetize.^ Beyond that, inconsistencies in how agencies monetize 
benefits cast serious doubt on the usefulness of the effort. EPA values each life saved through its 
regulations at $9. 1 million,"^ but lives saved by Department of Transportation regulations are 
worth closer to $6 million. ' Cost-benefit figures therefore provide the aura of precision and 
rationality. It may be possible to asses the impacts of regulation in monetary terms, although 
pre-regulation estimates of costs often turn out to be inflated in retrospect. Efforts to monetize 


^ EmTl, Protection Agency', The Benefits and Costs of the Clean Air Act from 1990 to 2020, 5-18 to5-19 (Table 5-4) 
(Miir. 2011), available at htmi./AYww.eDa.Eoy/oar/sectSn/feb i l/fullreDort.ndf . 

■ EmTl, Protection Agency', Regulatory^ Impact Analysis of the Final Clean Air Mercury Rule 10-5 (Miir, 2005), 
available at httj3://wwv.epa.gov/ttnecasl/re.i;data/''RIAs/iacrcu.iY ria final. pd f. 

^ See, e.g., U.S. DepT of Transportation, National Highway Traffic Siifety Administration, Final Regulatory Impact 
Analysis, Corporate Average Fuel Economy and CAFE Reform for MY 2008-201 1 Light Trucks, at Vni-64 to VIII- 
65 (March 2006), available at 

l-itjD:/./ww.i4itsa.dot,uov/statiQfiles/DQT,/NHTS/VRi!leiii;ikmLV'Rines/.Associated%2QF!lss,^2006 FRlAPubhc.pdf . 

^ Em'tl. Protection Agency', Regulatory Impact Analysis (RLA) for Existing Stationary Compression Ignition Engines 
NFS! LAP, Final Draft 7-6 n. 8 (Feb. 2010). available al h!}D:.7ww\v.cpa.aov7tiTi/^auv/ncc/iicc ncslian ria2 -i7- 
lOixl f. 

'■ U.S. DepM ofTransporlalion. National Highway Traffic Safety Administration, Final Regulatory Impact Analysis, 
FMVSS2I6, Upgrade Roof Crush Resistance 121 (Apr. 2009). 
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benefits often depend on arbitrary and unrevealed assumptions and serve to undervalue the social 
benefits of regulation as benefits that are difficult or impossible to quantify get ignored or 
downgraded in importance. 

Indeterminacy is inherent to cost-benefit analysis. We’ve been using cost-benefit as a 
regulatory guide star for 30 years, and so we’re in a position to draw that conclusion. The 
indeterminacy has been with us since day one, and it’s not going away. Efforts to reform the 
methodology will at best yield only marginal improvements in its ability to improve regulatory 
decision-making. Cost-benefit analysis attempts to distill a large and complicated body of 
information into a few numbers. The information on which the analysis is based is always full of 
uncertainty and imperfections. Data are never complete. Scientific conclusions are never 
certain. And the process of converting intangible values into monetary terms is fraught with 
unsolvable theoretical conundrums. Over time, methodological improvements may chip away at 
some of the ambiguity and uncertainty underlying every cost-benefit analysis. However, cost- 
benefit analysis will never be able to adequately measure the net benefits of a particular 
regulatory option, much less to allow for a meaningful comparison of the net benefits of several 
regulatory options to determine which maximizes net benefits. 

That doesn’t mean that potential costs and benefits shouldn’t be weighed by regulatory 
agencies. Agencies ought to assess costs and benefits. But such estimates ought to provide only 
one piece of the puzzle, and should be used with caution and with an acknowledgment of their 
limitations. Supporters of cost-benefit analysis sometimes seem to believe that these flawed 
estimates can be the sole basis for determining whether regulations are workable or worthwhile, 
as if a computer that has been fed all the details will just do the math and then crank out an 
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“optimal” decision for us. But if the data going into the computer are incomplete and flawed, as 
they inevitably are, the result of the process is bound to be flawed as well. 

To Improve the Regulatory System, the Protector Agencies Must be Re -Energized 

The real problem to which Congress should be directing its attention is not insufficient 
agency attention to cost-benefit methodologies. A recent series of catastrophic regulatory 
failures have brought to light the indisputably troubling condition of crucial regulatory agencies 
assigned to protect public health, worker and consumer safety, and the environment. The 
destructive convergence of funding shortfalls, demonizing political attacks, and outmoded legal 
authority has set the stage for ineffective enforcement and unsupervised industry self-regulation. 
From the Deepwater Florizon spill in the Gulf of Mexico that killed eleven and caused grave 
environmental and economic damage, to the worst mining disaster in 40 years at the Upper Big 
Branch mine in West Virginia with a death toll of 29, the signs of regulatory dysfunction abound. 
Peanut products tainted by salmonella, glasses contaminated by cadmium sold to children at fast 
food restaurants. Code Red smog days when parents are warned to keep their children indoors, 
the recall of widely used pharmaceutical drugs found to create risks of heart failure — all reflect 
agencies unable to do their jobs and companies that put economic self-interest above operating in 
ways consistent with the public interest. 

More analysis — including analysis aimed at increasing regulatory report card scores — 
will not fix these flaws in the current regulatory system. If anything, more analysis only makes 
things worse by needlessly slowing agencies down without demonstrably improving the quality 
of their regulatory decisions. The Occupational Safety and Health Administration’s (OSHA) 
recently issued Cranes and Derricks Rule provides a concrete illustration of the pernicious 
effects of “paralysis by analysis.” Beginning in the mid-1990s, industry itself began petitioning 
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OSHA for stronger and more comprehensive regulations and in July of 2004 a eommittee of 
industry, labor, and government representatives reached agreement on a draft proposed rule. 
Nevertheless, an understaffed, under-resourced, and over-stretched OSHA was not able to Issue a 
final rule until August of 2010 — more than 6 years later — because it was tied up by several 
burdensome analyses.'’ By OSHA’s own estimates, every year the rule remained stuck, 53 
people died and another 155 were injured unnecessarily.’ 

Yet, proponents of cost-benefit analysis remain focused on perfecting formulas, 
assumptions, models, and data sets. But elaborate efforts to monetize heart attacks do not 
prevent heart attacks; EPA’s air pollution regulations do. A rigorous cost-benefit analysis does 
not prevent workplace deaths; an energized and unencumbered OSHA does. 

If we want to fix the regulatory system, we should instead focus on finding ways to help 
agencies effectively achieve their statutory missions of protecting people and the environment. 
Here are some places to start: 

• Providing agencies with the resources they need. One of the reasons that 

regulatory agencies cannot fulfill their statutory missions is that financial resources 
and available personnel have been reduced or maintained at constant levels in reeent 
years. This has been occurring as the agencies’ missions have become more 
complex, thus forcing these agencies to effectively do more with less. And the 
situation is getting worse, not better. Just last week in an editorial published in the 
Washington Post on March 24, 201 I, William Ruckelshaus and Christine Todd 


^ Cranes mid Derricks in Constniction, 75 Fed. Reg. 47906 (Aug. 9, 2010) (to be codified at 29 C.F.R. pt. 1926), 
available at littDi/Avww.os tui .gov.TedReg oslia pdf/FF,D2Q 100809.pd f. See Catherine O’Neill et af The Hidden 
Human and Knvirtmmenlal Costs of Regulatory Delay 13-16 (Cir. for Progressive Rcfonn. While Paper 907, Ocl. 
2009), available at ht(i>:/./wiv\v.l)roeivssivcrcro n n.org/arliclcs/Coslol19 c iav 907 .pdf. 

Cranes and Derricks in Conslmclion, 73 Fed. Reg. 59714, 59884 (proposed Ocl. 9, 2008) (lo be codified al 29 
C.F.R. pi. 1926); O’Neill cl al., supra note 6, al 15. 
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whitman, who served as EPA Administrators under Republican Presidents Nixon, 
Reagan, and George W. Bush, recognized the threat and decried House proposals to 
cut EPA’ s budget by nearly a third, which they said would “impede [EPA’s] ability to 
protect our air and water ” 

• Providing agencies with enhanced legal authority. For many regulatory agencies, 
the statutes under which they operate have not been reviewed or refreshed in decades. 
The intervening years have revealed shortcomings in those statutes while new public 
health, safety, and environmental issues that were not initially addressed by the 
original statutes have emerged. Again, the warnings of Administrators Ruckelshaus 
and Whitman ring true. “Amid the virulent attacks on the EPA driven by concern 
about overregulation,” they noted, “it is easy to forget how far we have come in the 
past 40 years. We should take heart from all this progress and not, as some in 
Congress have suggested, seek to tear down the agency that the president and 
Congress created to protect America’s health and environment.” The same holds true 
for the other health, safety, and environmental watchdog agencies. 

• Freeing agencies from unnecessary analytical requirements. Over the past few 
decades, the rulemaking process has become encumbered by a growing number of 
analytical requirements. These analytical obstacles draw upon agencies’ already 
stretched resources and distract them from focusing on their regulatory missions 
without meaningfully improving the quality of agency decision-making. 

The Traditional Administrative Rulemaking Process is Already Designed to 
Identify the Need for Rfgut.atton and Account for Regut.atory Impacts 
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Although there is no evidence to support the charge that agencies routinely churn out ill- 
advised and counterproductive proposals, current law provides ample opportunities to fix those 
problems without heaping on agencies already stretched to the limit more onerous analytical 
responsibilities. A regulatory proposal is just that — a proposal. It reflects the best efforts of an 
agency to devise a regulatory solution to some environmental, health, or safety threat that is 
supported by applicable law and available science. The solution is the result of a broad inquiry 
into the nature of the threat and the available remedial options that is conducted by an 
interdisciplinary group of agency experts and policymakers. 

Despite these best efforts, sometimes an agency overlooks some crucial issue when 
developing a rule. This is why, under traditional Administrative Procedure Act (APA) 
rulemaking, a regulatory proposal is meant to start the discussion, not end it. Indeed, the agency 
must solicit and actually consider comments it receives from the public on the proposal. If the 
agency discovers during the comment process that it has strayed beyond its statutory authority, 
neglected relevant considerations, or misunderstood the science on which it based its proposal, 
the APA requires the agency to revise the rule accordingly before finalizing it, or not adopt the 
rule at all. This is not some hollow exercise. Rather, it is strictly enforced by federal courts 
whenever those affected by a final rule challenge it in court. If the reviewing court finds that an 
agency ignored some relevant public comment without adequate explanation, it can vacate the 
rule and send the agency back to the drawing board. This prospect creates strong incentives for 
agencies to diligently consider all relevant infonnation during development of the rule. 

In essence, efforts to reform regulatory analysis through enhanced cost-benefit analysis 
ignore this well-calibrated process. Instead, these efforts would require agencies to embark on a 
time-consuming, resource-intensive, and ultimately fruitless search to uncover every impact that 
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a rule might have at the very beginning of the rulemaking process. This will not improve 
regulatory decision-making. At best, it wastes agencies time and resources. At worst, it stops 
the whole rulemaking process dead in its tracks. 

IV. Regulations Can Be Revised Through Incremental Adjustments 

Even if agencies get it wrong during initial rule promulgation, the regulatory process 
provides ample avenues for those affected by the regulation in unintended or counterproductive 
ways to seek relief from the agency. Agencies spend much time and effort attempting to 
rationalize significant draft regulations before they are proposed and adopted. These efforts are 
certain to fail at times because of methodological and informational problems. It therefore 
makes more sense for regulators to make incremental adjustments to regulations at the “back- 
end” of the administrative process, by relying on exceptions, time extensions, variances, and 
waivers, rather than continuing the effort to rationalize regulation at the “front-end” of the 
process.* By focusing on the back-end, administrators have the opportunity to adjust regulations 
in light of their actual impact, as compared to the unavoidable and significant guesswork used in 
front-end analysis such as cost-benefit analysis. 

Congress has generally authorized most agencies to make these back-end adjustments. 

A back-end adjustment process has several advantages over efforts to craft a perfect and 
omniscient regulation at the outset. First, it permits agencies to preserve relatively stringent 
baseline risk-reduction standards while still accommodating concerns that the application of 
these stringent rules will cause irrational or unfair results in particular cases. Regulators can 
make case-by-case adjustments instead of initially watering down standards in anticipation that a 
general rule may be counterproductive or irrational in some circumstances. Second, a back-end 

^ See Robert L. Glicksiran & Sidney A. Shapiro, Improving Re^utalion I'hroifgh Incremental Adjustment. 52 U. 
Kan. L. Rkv. 1179 (2004). 
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process addresses the delays caused by analysis requirements and the difficulty of undertaking 
analysis in light of informational and methodological problems. The availability of these 
adjustments can avoid delay in the issuance of a rule of widespread applicability because an 
agency can promulgate a rule and rely on regulated entities to alert it to implementation 
problems by filing individual requests for relief. Further, a back-end process gives regulated 
entities a strong incentive to produce evidence that an adjustment in a rule is justified. A process 
that relies on back-end adjustments to fix regulatory flaws places on those most likely to possess 
information bearing on how regulation has produced unintended consequences or unfair 
treatment — the regulated community — the incentive to bring that information to the agency’s 
attention. Unlike rulemaking, in which regulators must attempt to anticipate problems before 
they occur as they write general rules, incremental adjustments permit regulators to consider 
concrete problems, one at a time, in the context of specific circumstances. The back-end process 
allows agencies to make adjustments in response to circumstances that they did not anticipate 
when they wrote a rule. 

Third, a back-end adjustment process can increase the legitimacy of the regulatory 
program that contains the back-end process by reducing the frustrations likely to result from the 
application of regulatory requirements in ways that produce harsh or anomalous results. 

Finally, but hardly least of all, a back-end process is one of the ways that regulators can 
take costs into account. A back-end adjustment process that authorizes hardship-based 
adjustments makes cost a relevant consideration without relying on a cost-benefit test that yields 
a misleading impression of analytical precision. 
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It is important that agencies be accountable for how they make back-end adjustments.'^ 
But this method of improving regulation makes far more sense than endless attempts to perfect 
the cost-benefit analysis that occurs at the front-end of the regulatory process. 

V. Regulations Can be Revised Through Subsequent Rulemakings 

At least one other aspect of current regulatory practice functions as a device for weeding 
out flawed regulations. On occasion, a rule imposes burdens without providing much benefit. 
Much more frequently, the agency discovers that the rule is not strong enough, as illustrated by 
EPA’s 1973 Clean Air Act regulation that required refineries to reduce the amount of lead in 
gasoline by about 80 percent. Subsequent epidemiological studies confirmed that the rule 
significantly reduced blood lead levels in children, preventing countless cases of learning 
disabilities and impaired brain development, while imposing relatively little cost on refineries. 
These studies also confirmed that the 1973 lead rule did not go far enough in protecting the 
public health (specifically, in protecting children exposed to lead from cognitive impairment). In 
1 985, EPA tightened the standard even more, and Congress eventually banned the use of lead as 
a gasoline additive in the 1990 Clean Air Act Amendments. EPA was convinced in 1973 that 
lead was harmful to public health, but because of the state of the science at the time, it 
underestimated its adverse impacts. When EPA acquired that knowledge later, it amended its 
rules to better advance the fundamentally precautionary mission of the Clean Air Act.*° 

Problematic regulations of either variety (excessive or inadequate) can be fixed through 
subsequent rulemaking actions. Some statutes contemplate this eventuality, requiring periodic 

^ See Sidiiev A. Shapiro & Robert L. Glicksiruia TheAPA and the Back-End of Regulation, 56 ADMIN. L. Rev. 1139 
(2004). 

'“.See Ethyl Corp. v. EPA, 541 F.2d 1, 13 (D.C. Cir. 1976) (eiibimc) (“A statute allowmg for regulation in tlie faee 
of danger is, nceessarilv. a prceaulionaiy slalule. Regulatory aeliou may be taken before the (hrealetied harm 
oeeursi indeed, (he very' exislenee of sueh preeaulionary legislation would seem to demand lhal regulalory aeliou 
preeede, and, oplimallv, prevent, the pereeived threat. As should be apparent, the 'will endanger' language of [the 
Clean Air AelJ makes il sueh a preeaulionary slalule.”). 
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review and revision of existing rules. For example, the Clean Air Act requires EPA to revisit its 
National Ambient Air Quality Standards (NAAQSs) for criteria pollutants at least once every 
five years and revise them as necessary. The same law requires EPA to assess whether 
technology -based controls on emissions of hazardous air pollutants provide sufficient health 
protection. If not, EPA must recommend supplemental regulation to Congress and adopt it if 
Congress does not. 

Efforts to reform cost-benefit analysis ignore the regulatory system’s capacity for self- 
correction. If anything, these efforts undermine this capacity by tying up the regulatory process 
in knots. Instead, the cost-benefit analysis reform movement aims at perfection at the front end 
of the regulatory process, with the inevitable result that agencies will not have time and resources 
to issue many regulations and those that are issued will tend to underprotect health and safety as 
regulatory benefits are undervalued. If cost-benefit analysis had been the order of the day in 
1973, it is highly unlikely that EPA would have been able to issue its important lead rule due to 
the absence of an established connection between automotive lead emissions and health 
problems in children." If the cost-benefit analysis reform movement succeeds, one can only 
imagine all the important future safeguards that will be unnecessarily delayed and diluted. 

I will close by returning to the recent admonitions of Administrators Ruckelshaus and 
Whitman. They recognized what is obvious but that nevertheless bears repeating: “Our country 
needs today what it needed in 1970: a strong, confident, scientifically driven, transparent, fair 
and responsible” set of protective agencies such as EPA. “Congress should help America 

” Frank Ackemiim, Lisa Heinzerling, & Rachel Massey, d/yi/vtng Cost-Benefit to Past Decisions: Was 
Environmental Protection Ever a Good Idea?, 57 Admin. L. Rev. 153, 161 (2005) (“Thus, the cost-benefit analysis 
of the 1 980s phase down ot lead in gasoline would nol have been possible in Ihe absenee ot Ihe more iinporlanl 
197()s-era regulation — which was nol itself based on eosl-benefil analysis. Had we wailed in (he 197()s. as some 
argue we should do in poliey disputes today, for eost-benefil analysis to show us the way. we might still be waiting 
now.”). 
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achieve that ” They also fired a shot across the bow of those antagonistic to these goals: “The 
American public will not long stand for an end to regulations that have protected their health and 
quality of life.” 


Mr. Coble. Gentlemen, as I said, we try to apply the 5-minute 
rule to us, so if you all can keep your response to a terse manner, 
we would appreciate that. 

Failure is unfortunate. Some people indicate that those of us who 
want to refine the regulatory system or improve it, including Presi- 
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dent Obama has said that, we are not trying to compromise safety 
in doing this. At least that is my take on it. I think that needs to 
be fully appreciated. 

Dr. Ellig, in 2008, the average Bush administration agency score 
on your report was a failing 27.31 out of 60 possible points. In 
2009, the average Obama administration score was also a failing 
mark of 27.02 out of 60. Is it fair then to conclude that the failure 
of good decisionmaking is a systematic problem across the Repub- 
lican and Democratic administrations? 

Mr. Ellig. Yes. Statistically, there is no difference between those 
two figures. We are getting about the same results for both years. 

Mr. Coble. Thank you, sir. 

Professor Glicksman, in your book. Risk Regulation At Risk, you 
write that agencies are more acquainted with the day-to-day de- 
tails and difficulties of regulatory decisions than the communities 
of inquiry that operated the White House, Congress and the Fed- 
eral judiciary, which are not specialized in the same way. Because 
agencies are much less accountable than the other branches of gov- 
ernment, should we not be doing even more to ensure that their de- 
cisions are made transparently and on the record? 

Mr. Glicksman. I certainly support transparent decisionmaking 
by agencies, and the tools that are provided by the Administrative 
Procedure Act to ensure transparency. Beyond that, both of the 
other branches of government, the Congress and the courts, do 
have ways of holding agencies accountable through, for example, 
amending statutes that Congress decides agencies have imple- 
mented in ways that don’t conform to congressional intent. The 
courts are often called upon to review regulations issued by agen- 
cies, and they have the responsibility, as well as the authority, to 
overturn regulations that lack adequate justification or are not sup- 
ported by scientific evidence. And they do so on a regular basis. 

Mr. Coble. Thank you, sir. 

Dr. Williams, under your view, agencies really don’t have an in- 
centive to seek and heed input from those who suggest no regula- 
tion is needed. Instead, all of the incentives are to listen and those 
who want more regulation, whether or not more is needed. What 
are the most important ways in which we can reform the APA to 
restore agencies’ incentives to solve problems rather than just regu- 
late for the sake of regulating? 

Mr. Williams. Thank you for the question. I think what happens 
now is that agencies are rewarded with budgets for passing regula- 
tions, and they are not penalized for passing regulations that are 
ineffective or unwanted. I think the first thing you have to look at 
are what are the incentives that need to be changed with agencies. 
I would start with the process that I outlined earlier where you 
separate out agencies, defining the problem and taking public input 
on what the problem is before they go ahead and make a decision. 
But somewhere down the road, you are going to have to address 
the incentives that agencies face, and particularly with budgets. 

Mr. Coble. Dr. Ellig, let me try to beat that red light. The first 
and most important step in any good decisionmaking is to identify 
the problem one is attempting to solve. Is it really true that you 
found agencies did the worst job of all at this most basic step of 
decisionmaking? 
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Mr. Ellig. Yes. The average score on our criterion for evaluating 
the systemic problem was around a 1.8 out of 5. So on average, 
that was with the worst criterion. And about half of the time we 
found it wasn’t really that well addressed at all. 

Mr. Coble. I will now recognize the gentleman from Tennessee 
for 5 minutes. 

Mr. Cohen. Thank you, Mr. Chairman. 

Has the Mercatus group ever studied the Justice Department’s 
regulations or policies? 

Mr. Williams. Not to my knowledge, sir. 

Mr. Ellig. We included several Justice Department regulations 
in our regulatory report card if they were proposed in 2008 or 2009. 
And at some time in the past, one of many independent scholars 
at Mercatus may have done some comments on Justice Department 
some time in the past 20 years. 

Mr. Cohen. Mercatus has been around for 20-30 years; right? 
When did you start your report card? 

Mr. Ellig. This regulatory report card project started with the 
year 2008 a couple of years ago. A precursor project examined the 
quality of analysis of all regulations issued by the Department of 
Homeland Security since its inception. And that was done a couple 
of years ago. 

Mr. Cohen. So you had a report card in 2008? 

Mr. Ellig. We started with the proposed regulations for 2008. 

Mr. Cohen. Did you have a report card for the year 2008? Or 
was the first report card in 2009? 

Mr. Ellig. Just to clarify, we don’t have really a single document 
that we call a report card. The project is a wide variety of evalua- 
tions of individual regulations which are all available on our Web 
site, along with a set of notes which justify each score. This is the 
most transparent way this kind of a project has ever been done in 
the United States. We have that. 

Eor the past 2 years, we have produced a paper summarizing the 
results for the year, and we have a paper from 2008 and a paper 
from 2009 that also compares the 2008 and 2009 results. 

Mr. Cohen. Who is on your board of directors? 

Mr. Ellig. Who is on our board of directors? 

Mr. Cohen. Yes. 

Mr. Ellig. I can name a few folks. Honestly, I can’t name them 
all. 

Mr. Cohen. Name a few of them. 

Mr. Ellig. I can name a few. Vernon Smith, Nobel Laureate in 
economics. 

Mr. Cohen. Erom industry, which industry people are on your 
board? 

Mr. Ellig. Erom industry, the last time I looked, Charles Koch 
was on the board. Industry, I am not sure if a former Eed vice 
chairman counts as industry. 

Mr. Cohen. No, it doesn’t. 

Is Charles Koch your largest funder? 

Mr. Ellig. I honestly don’t know. I don’t care and it is not 

Mr. Cohen. I care. The issue is I care. Is he your largest funder? 

Mr. Ellig. I do not know. 

Mr. Cohen. Is he one of your largest funders? 
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Mr. Ellig. I do not know, and it is not relevant to my work. On 
our Web site, we have a policy on the independence of research 
from funding. 

Mr. Cohen. Dr. Williams, do you know who your largest funders 
are, the largest funders of the Center? 

Mr. Williams. No, sir. I am also on the analytical side, and we 
have a very strict separation of who funds us and how we choose 
our analysis. I basically do analysis that I care deeply about, and 
most of it stems from the work I did at the Food and Drug Admin- 
istration. And I still work intensively on those issues because I care 
about the food and safety issues. 

Mr. Cohen. The issues of food and drug, we have had spinach 
scares and we have had lettuce scares and chickens we had to get 
rid of and the Asian flu. We have had other problems with food. 
Are you concerned that a lessening of regulation might not subject 
the American people to more and more contaminated foods? 

Mr. Williams. No, sir. 

Mr. Cohen. You are not? 

Mr. Williams. My concern is when too often the agency either 
wants to or is forced to regulate before we have solutions to prob- 
lems. What that means is we end up putting out regulations, peo- 
ple have to comply with those regulations, and we don’t solve the 
problem. For the 27 years I was in the Food and Drug Administra- 
tion, we did not see food-borne illness decrease by one food-borne 
illness. We put out a number of regulations that were ineffective. 
What I am here to do today is to try to get effective regulations; 
not more, not less, effective regulations. 

Mr. Cohen. Professor Glicksman, Robert Reich, former Labor 
Secretary, said that those who argue that regulations kill jobs ig- 
nore an important fact: lack of adequate regulation kills people. 
You made some reference to that in studying effects, whether it is 
just dollars and cents or human lives. Do you think some of the 
changes in regulations that have been proposed might affect peo- 
ple’s lives in having more contaminated food? 

Mr. Glicksman. Certainly, I think watered-down regulation or 
repeal of regulation is likely to have adverse effects on the public 
health and safety and the environment, which can translate into a 
downturn in economic productivity. There is often a dichotomy that 
is set up between regulation that is designed to protect the health 
and safety and the environment and economic productivity. I think 
it is a false dichotomy. They go hand in hand in the same direction 
in many instances. 

Mr. Cohen. Thank you. 

Mr. Coble. I thank the gentleman. 

The distinguished gentleman from South Carolina, Mr. Gowdy, is 
recognized for 5 minutes. 

Mr. Gowdy. Thank you, Mr. Chairman. 

Dr. Ellig, in the course of 5 minutes, it struck me that both the 
motive and the methodology have been questioned by some of my 
distinguished colleagues from the other side. If you would, please, 
would you tell us about your methodology? Why who is on your 
board doesn’t influence your research? Go ahead and say what you 
wanted to say when my colleague from Tennessee was questioning 
you. 
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Mr. Ellig. Basically, the way we are set up at Mercatus, fund- 
raising is separate from research. And those of us who do research 
have the freedom to call them as we see them. That is what I like 
about working there. If it were organized in some other way, I 
probably wouldn’t want to work there. 

Now, as far as the scorecard goes, let me take off on the use of 
the word “subjectivity,” since the Ranking Member mentioned that 
in his introduction of our scorecard. 

We said in the paper we wrote that a critic might claim that this 
evaluation method is subjective and we have gone to great pains 
to try to minimize the problems that might occur as a result of 
that. And the actual word we used in our 2008 paper was “inter- 
subjective,” which is a term from philosophy of science which 
means different people may be doing an evaluation and rendering 
a subjective opinion, but what counts for a 2 when I look at a regu- 
lation is the same as what counts for a 2 when Richard looks at, 
what counts for a 2 when our colleague John Morrall, a 29-year 
veteran of the Office of Information and Regulatory Affairs, looks 
at it, and so forth. So we try to get a common understanding of 
what types of performance in these analyses counts for what kind 
of a score. 

Then we put the scoring notes and the scores all up on our Web 
site so anybody can go look at it, and we welcome other folks to 
look at it, to read it and dialogue with us. And if they think we 
missed something, tell us. That is what it is about in academia. We 
won’t get upset; but just tell us what the specific issue or the prob- 
lem is. 

Mr. Gowdy. Well, it strikes me that you gave equally abysmal 
scores to both Administrations. So the notion that you are biased 
in favor of one Administration or the other, I would be embar- 
rassed at either score that was given. Let me ask you this to hope- 
fully end on a happy note. The Department of Justice got the high- 
est ranking that you gave. What are they doing that the Social Se- 
curity Administration is not? 

Mr. Ellig. I don’t think we have enough data yet to say well, 
this agency generally does a good job; and this agency generally 
does a bad job. We have a small number of agencies that will issue 
six or seven or 10 big regulations a year. But a lot of the other ones 
will be one or two or three, and there are different sub-entities 
within the agencies, some of which may do better or worse analysis 
than others. I don’t know that we can generalize from our results 
at this point to say this agency does a good job and this agency 
does a not-so-good job. 

Mr. Gowdy. Mr. Chairman, in a rare move, I yield back the bal- 
ance of my time. 

Mr. Coble. I thank the distinguished gentleman from South 
Carolina. 

The distinguished gentleman from Michigan is recognized for 5 
minutes. 

Mr. Conyers. Thank you, Mr. Chairman. 

Members of the panel, I appreciate your comments. 

Do you have any familiarity with legislative Acts that are at- 
tempting to reform the regulatory process by the Congress? 
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Mr. Ellig. My understanding is all kinds of stuff have been in- 
troduced. 

Mr. Conyers. Anything in particular? 

Mr. Glicksman. I am somewhat familiar with the proposed 
REINS Act. 

Mr. Conyers. What about you, Dr. Williams? 

Mr. Williams. I also heard about the REINS Act as well. 

Mr. Conyers. Have you heard anything good about it? 

Mr. Williams. My understanding, sir, is that Congress, a num- 
ber a years ago, voted itself the right to turn back regulations, and 
that they have used that authority one time in something like 15 
years. And that the Reins Act would reverse that. 

Mr. Conyers. Do you have any particular recollection. Dr. Ellig, 
about any proposals currently before the Congress, in this 112th 
Session? 

Mr. Ellig. I have heard of the REINS Act. I have heard of a 
piece of legislation that essentially took President Clinton’s Order 
12866 and wants to write that into law so it is a requirement for 
all agencies rather than just an executive order. 

I remember hearing a few days ago there was a piece of legisla- 
tion introduced that would have — that would essentially almost 
overturn a key Supreme Court precedent on direct wine shipment 
and give the States more latitude in how they choose to regulate 
interstate commerce in alcohol. So, yes, a lot of different things 
have been introduced. 

Mr. Conyers. For experts in regulatory reform, your attention to 
some of the excessive amounts of time we spend on the debates is 
a little bit disturbing to me. This is the fourth hearing — this is the 
fifth hearing on regulatory reform. Have you ever heard anything 
about our hearings in this Committee, in the Judiciary Committee? 

Mr. Ellig. I am aware of one previous hearing. I have to say as 
a regulatory specialist who used to work in a Federal regulatory 
agency where the general — a lot of issues of regulation I work on 
rarely get much attention in Congress, so I am delighted that it is 
finally going to get some attention. 

Mr. Williams. My expertise is actually in regulatory agencies for 
27 years. And the things I testified about today are things I came 
to realize over really three decades. I pay some attention to the 
bills that are going on in Congress. But also as an analyst, I am 
familiar with the fact that many bills are introduced but not 
passed. 

So my concern today is to try to fix the problems where agencies 
are making decisions first, and then getting stakeholder input. 

Mr. Conyers. Well, I am a little perplexed about your expertise, 
but in the storm that is going on these first 3 months about regu- 
latory reform, this Committee has spent more time on this subject 
than anything else. You spent your careers working in the field. 
And now we meet here this afternoon and you know almost little 
or nothing about what is being contemplated. How can we be ex- 
pected to take your advice seriously if you don’t even have any 
knowledge of what we are doing to try to improve the regulatory 
process of which you complain pretty strenuously? 

Mr. Williams. It is not my understanding that we were called 
here to talk about individual, proposed statutes. In fact, because of 



100 


our status, we actually can’t advocate for or against any particular 
law. 

Mr. Conyers. You can’t? Well, let me ask you this: Who are the 
agencies that are doing so poorly that they get zero or one ratings? 
Are you able to reveal that? 

Mr. Ellig. Well, it is all revealed on our Web site. So, it is cer- 
tainly already public knowledge. 

Mr. Conyers. Yes, but you are here in front of me right now. 
Let’s talk about it. 

Mr. Ellig. The zeros or the ones were just on one criterion which 
was 

Mr. Conyers. Mr. Chairman, I ask unanimous consent for one 
additional minute. 

Mr. Coble. Without objection, one additional minute. 

Mr. Conyers. Thank you very much. 

Mr. Ellig. The zeroes and the ones were on one particular cri- 
terion which was definition of the systemic problem. 

Mr. Conyers. Just name the agencies you are talking about? 

Mr. Ellig. An agency that had a lot of regulations that were 
zeros or ones was Department of Health and Human Services. 
Typically in the annual regulations, the Department issues recal- 
culation of Medicare payment rates and various other things like 
that. 

Mr. Conyers. Thank you for that one. 

Name a second one? 

Mr. Ellig. Let’s see. In 2008, there was a Social Security Admin- 
istration regulation on setting the time and place for appearances 
before administrative law judges that just didn’t say anything 
about the issue. 

Mr. Conyers. What agency are you referring to, sir. 

Mr. Ellig. Social Security Administration. 

Mr. Conyers. You have got a list of maybe about 15, I will just 
ask you to submit them for the record. 

Mr. Ellig. Each year there were about half of the regulations 
that got zeroes or ones, so that would be about 20 to 25 a year. 

Mr. Conyers. Right. Thank you very much, Mr. Chairman. 

Mr. Coble. You are welcome. The distinguished gentleman from 
Georgia is recognized for 5 minutes. 

Mr. Johnson. Thank you, Mr. Chairman. With all due respect, 
I must wonder why it is that we are holding this hearing today. 
Purportedly, it is going to give the Members an opportunity to dis- 
cuss ways in which Federal agencies can improve their grades on 
the regulatory report cards issued by the Mercatus Center. And 
this Mercatus Center was founded by Rich Fink, correct? Do you 
know who Rich Fink is? 

Mr. Ellig. Yeah, yes. I know who Dr. Fink is. 

Mr. Johnson. And Dr. Fink is former president for the Koch 
Family Foundation , isn’t that correct? 

Mr. Ellig. I don’t know what title, I know he works for Koch In- 
dustries. I don’t know what other titles he might have accumu- 
lated. 

Mr. Johnson. Now Koch Industries is led by the brothers David 
and Charles Koch, and it is a $100 billion per year conglomerate. 
Isn’t that a fact? 



101 


Mr. Ellig. It is a big company, yes. 

Mr. Johnson. The second largest private industry in the United 
States of America. 

Mr. Ellig. Yes. Sometimes the news reports say second, some- 
times they say first. 

Mr. Johnson. A $35 billion fortune that the brothers control, and 
out of that 35 billion, isn’t it a fact that they contributed about $30 
million to the George Mason University, much of which went into 
funding for the Mercatus Center? 

Mr. Ellig. I don’t know because we keep research separate from 
fundraising, and I am on the research side of 

Mr. Johnson. You wouldn’t disagree with that, would you? 

Mr. Ellig. I am saying I do not know. 

Mr. Johnson. Okay, well, I will tell you if it is true, and I have 
reason to believe that it is, doesn’t that put the credibility of the 
Mercatus Center and its report card at issue? Is that something 
that the public should be able to understand who is doing the grad- 
ing and perhaps the fact that they are grading is influenced by 
their monetary interest in what they are grading? 

Mr. Ellig. Well, I don’t pay any attention to it. And as far as 
whether that has any effect on the credibility, I am quite happy to 
have the credibility of this project stand on the quality of the re- 
search results we produce, the quality of the process we put to- 
gether, and the reviews and comments that we get as we submit 
the papers from this project for peer review in academic journals. 

Mr. Johnson. Well, Dr. Williams, you have said pretty much es- 
sentially that agencies justify their very existence by issuing regu- 
lations. And oftentimes, there is no justification for the regulation 
that they promulgate. And you are saying, or can it be said also 
that legislators justify their existence by introducing legislation 
some of which may not be prudent, is that correct? 

Mr. Ellig. Well, what I know about is regulatory agencies. I 
have 27 years in the Food and Drug Administration. 

Mr. Johnson. Do you mean to tell me that you think that the 
average agency employee comes to work every day with the sole 
purpose of deciding what kind of regulation I am going to propose 
today? 

Mr. Williams. I can’t testify to all agencies. I know that the em- 
ployees of the Food And Drug Administration are some of the finest 
people I have ever meet. However, I do know 

Mr. Johnson. It is the Department of Energy, though, that you 
really want to get at. Tell me this. If you really want to get, since 
the Kochs are in the energy business, have a lot of concerns in the 
energy business, you are trying to make it more difficult for the 
Department of Energy and other departments to issue regulations 
which would govern the conduct of this for-profit corporation. 

Mr. Williams. Far from it. What I am concerned about, the same 
thing I was concerned about throughout my entire career is that 
we get effective regulations, that we not have ineffective regula- 
tions that crowd up compliance with the effective ones so that we 
can actually solve our social problems. That is my concern. 

Mr. Johnson. Thank you, sir. 

Mr. Coble. We have been joined by the distinguished gentleman 
from California, Mr. Gallegly. 
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Mr. Gallegly. Thank you very much, Mr. Chairman. 

Dr. Ellig, could you mayhe give us a little assessment about what 
steps the Congress should take to ensure agencies consider the eco- 
nomic impact of all regulations, report regulations are imposed? 

Mr. Ellig. Well, I don’t think there is a silver bullet because the 
regulatory process is complex but there are some things that could 
help. One is to find some way to ensure that agencies actually do 
the analysis before they make decisions about regulations. 

Now, I know some folks say, oh, that is going to slow down the 
process, paralysis by analysis. I only know of one empirical inves- 
tigation that tried to figure out whether that is actually true or 
not. It is a classic little article by a gentleman named Tim Muris. 
He is a distinguished law professor at George Mason. He was 
chairman of the Federal Trade Commission. And back in the 1980s, 
he wrote an article in which he looked at Federal Trade Commis- 
sion rulemaking and pointed out that one of the most famous regu- 
lations in which the FTC took its time to develop a theory of the 
systemic problem and then investigate empirical evidence, the eye- 
glass rule to prevent bans on advertising of eyeglasses, the rule 
was done in 3 years. 

Another rule, a famous rule the FTC issued that was not really 
accompanied by a very good systemic theory of what the consumer 
harm was and didn’t have much empirical analysis and was essen- 
tially based on some anecdotes, was the FTC funeral rule. It took 
10 years to issue this thing. So I would suggest that maybe, just 
maybe, agencies can sometimes do a quicker job or at least a better 
job of regulating when they actually have to take the steps to un- 
derstand what they are doing before they do it. 

Mr. Gallegly. When you mentioned the concern for slowing 
down the process, I am not sure that that is necessarily a bad 
thing sometimes. In fact, it would almost appear to me that with- 
out doing some of this analysis, it is almost a ready-shoot-aim type 
situation. And ultimately, the cure may be worse than the disease. 
So maybe there should be more of an attempt to understand the 
consequences, not that in the end the results may be the same, but 
at the same time, I would think that should be an integral part of 
the equation. I hope somebody agrees with that. 

Dr. Williams? 

Mr. Williams. I agree. I certainly agree and as I said in my tes- 
timony, I think that too frequently we are making decisions first 
without knowing what the impacts of those decisions are, we are 
basically regulating in the dark. 

Mr. Gallegly. Professor Glicksman. 

Mr. Glicksman. I would just remark the regulations which I am 
most familiar the ones issues by EPA and Federal land manage- 
ment issues the BLM, the Forest Service and the Park Service are 
not characterized by lack of identification of regulatory objectives. 
It is quite clear when you read the preamble to an EPA regulation 
that seeks to control emissions of a cancer-causing pollutant that 
what the Agency is trying to do is to limit exposure to dangerous 
chemicals emitted by companies that, absent regulation, have little 
or no incentive to control their emissions in ways that will increase 
the regulatory compliance costs. 
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When an agency like the Forest Service issues regulations, it is 
quite clear that what they are trying to do is to enhance rec- 
reational opportunities for people like hikers, hunters, fishermen. 
So I just, in my experience, have not seen this problem of regu- 
lating without understanding what the objective of the regulation 
is going to be. 

Mr. Gallegly. I don’t know that any of us would really object 
to the objectives that you have just identified with. However, there 
sometimes are other issues that the economic impact could be more 
applicable than maybe the examples that you used. 

Thank you very much, Mr. Chairman. 

Mr. Coble. You are indeed welcome, sir. We have been joined by 
the distinguished gentleman from New York, Mr. Reed. You are 
recognized for 5 minutes. 

Mr. Reed. I am going to yield at this point in time, Mr. Chair- 
man. 

Mr. Coble. Thank you, sir. Gentlemen, we thank you for your 
testimony today. Without objection, all Members will have 5 legis- 
lative days to submit to the Chair additional written questions for 
the witnesses which we will forward and ask the witnesses to re- 
spond as properly as they can do so and that their answers may 
be made a part of the record. Without objection, all Members will 
have 5 legislative days to submit any additional materials for inclu- 
sion in the record. 

Again, gentlemen, thanks to each of you, and this hearing stands 
adjourned. 

[Whereupon, at 5:05 p.m., the Subcommittee was adjourned.] 
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APPENDIX 


Material Submitted for the Hearing Record 

Response to Post-Hearing Questions from Jerry Ellig, Senior Research Fellow, 
Mercatus Center, George Mason University; and Richard A. Williams, Director of 
Policy Research, Mercatus Center, George Mason University 


Questions for the Record 
Rep. Howard Coble 
Chairman 

Subcommittee on Courts. Commercial and Administrative Law 
For the Hearing on “Raising the Agencies’ Grades - Protecting the Economy, Assuring 
Regulatory Quality and Improving Assessments of Regulatory Need” 

March 29. 2011 


For Dr. Ellig: 

1 . During the hearing it was alleged that the Report Card Project Is entirely subjective and 
was not subjected to external review. Is this true? 

In Washington political debates, the tenn “subjective” is regularly equated with 
“arbitrary.” The term means something entirely different in the social sciences. In the 
social sciences, “subjective” refers to the perceptions and thought processes occurring 
inside a person’ s mind, as opposed to physical phenomena one can directly observe. 
Saying that thoughts and perceptions occur inside the human mind is not the same thing 
as saying that they can be dismissed as arbitrary. As we state in our June 2010 paper that 
outlines the full evaluation methodology, “As any professor who has graded papers 
knows, ‘subjective’ is not the same thing as ‘arbitrary.’”' Because we are human, we can 
understand each others’ subjective judgments and assess their reasonableness. 

The Regulatory Report Card is a qualitative evaluation, in which multiple experts assign 
scores to agency regulatory analyses based on consistent application of a common set of 
standards. We employ a qualitative evaluation to address the problem that the Office of 
Management and Budget pointed out in its 2008 report to Congress on the benefits and 
costs of federal regulations: “Objective metrics can measure whether an agency 
performed a particular type of analysis, but may not indicate how well the agency 
performed this analysis.”^ 

We point out that a qualitative evaluation runs the risk of being more subjective than a 
“check-the-box” evaluation. This is not a problem, as long as all of the scorers use a 
common frame of reference so that they share the same understanding of what kind of 
analysis deserves a 5, a 4, or any other score. Our paper explains the measures we have 
taken to mitigate this potential problem and ensure consistency in scoring. All scorers 
undergo training and practice evaluations to ensure that they share a common frame of 


’ Jerw Ellig and Patrick McLauglilin, “The QiialiW and Use of Regiilatorv' Analysis in 2008,” Mercatus Center 
Working Paper (June 2010), p. 8. h!!p: / ./inercat iis .org/ pub licnlK')ii'qua}iK'-aiid-!xse-r eguk ijo x\-a Ti;s j> sis-2008. 

” Office of Managctnciit and Budget 200H Rcporl lo Conff-ess on the Benefits and Costs of Federal Re^'idalions and 
Unfiwded Mandates on States. Local and Tribal Entities at 19. 
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reference. This shared understanding allows us to compare one regulatory analysis with 
another and identify best practices. The evaluation method is patterned after the Mercatus 
Center’s Performance Report Scorecard, a highly-regarded 10-year project that evaluated 
the quality of federal agencies’ annual performance reports produced under the 
Government Performance and Results Act.^ 

The Regulatory Report Card methodology and findings have been presented for review 
and critique at several professional and scholarly meetings attended by regulatory experts 
and government analysts, including the Society of Government Economists and Society 
for Risk Analysis. We presented the scoring methodology and results to several federal 
agencies and to the staff of the Office of Information and Regulatory Affairs, and we 
received many helpful comments and criticisms. We also received comments in one-on- 
one meetings with former OIRA and regulatory agency officials whose combined 
experience in the federal government exceeds 100 years. Several academic working 
papers based on the Regulatory Report Card data are under review at peer-reviewed 
scholarly journals. 

Finally, the fact that our findings are broadly consistent with what other regulatory 
scholars have found when they have evaluated regulatory impact analyses gives us more 
confidence that our method contains no serious flaws. 

2. You stated during the hearing, “at Mercatus, fundraising is separate from research." 
Would you care to elaborate on this point? 

Mercatus researchers are committed to the highest standards of academic quality and 
credibility for our research procedures and products. Mercatus emphasizes a free and 
open inquiry into understanding social problems and their remedies. Mercatus research 
projects are not directed research or research for hire, and financial supporters have no 
role in the selection or conduct of our research. We use internal and external review to 
ensure that any research product is constructed logically and credibly and that there is a 
thorough comprehension of existing economic literature. All conclusions belong solely to 
the author(s). 

While the credibility of the research any organization produces should stand on the 
quality and reliability of that research alone, more is often necessary. Mercatus strictly 
adheres to a stated Policy Regarding Independence of Research, wEich is as follows (and 
is available at httD:.//mercatus.ors:/about/'research-pojtcv) : 


For more informalion, see hnp://3i'ie rca ius.o rg'^ pi.iblicatio3!/ lO th-niiT}!inl-perror ni fl nce -repor1-scorecm'cF\vlucli- 
fcd crai-a gcii cics-b cst-uifo rm - p ii biic . 
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1 . The Mercatus Center engages in research and educational activities that 
advance the mission of the organization. This includes research and educational 
activities that may run counter to the interests of organizations and individuals 
that provide financial support to Mercatus. 

2. Mercatus accepts financial support from numerous sources for research and 
educational activities that support its mission, endeavoring to maintain a broad 
and diverse base of support. All Mercatus work product has to be capable of 
withstanding rigorous peer review of its quality and reliability. Therefore 
Mercatus financial supporters have absolutely no influence or control over the 
research design, methodology, analysis, or findings of Mercatus research projects, 
nor do they have influence or control over the content of educational programs. 
Offers of financial support predicated on such expectations are not accepted. 

3. Mercatus will not engage in research or educational activities that are co- 
sponsored with non-academic organizations that support Mercatus financially. 

4. The following paragraph wall be furnished to all organizations that provide or 
are considering providing Mercatus with financial support to ensure that 
supporters understand our commitment to independent research: 

The Mercatus Center is committed to the highest standards of academic quality 
and credibility and to ensuring that our work stands up to rigorous peer review. 
Mercatus scholars independently pursue a research agenda and educational 
activities that advance our mission. Mercatus docs not engage in research or 
educational activities directed or influenced in any way by financial supporters. 

To view our full policy regarding independent research, visit our Policy- 
Regarding Independence of Research Page on our Web site 
at httD://www.mercatus.ora/aboufresearch-poiicv . 

5. This policy and Mercatus practices are reviewed annually to ensure that 
Mercatus is in compliance with its provisions and faithful to its intent. 

For Dr. Williams: 

1 . Professor Glicksman suggests that we should allow agencies to err on the side of over- 
regulating at the outset, and then later modify promulgated regulations if they don’t 
actually work. 

a. Shouldn’t we demand that agencies get their regulatory decisions right to begin 
with? 


Yes. One of the best ways that agencies can accomplish this objective is to do a thorough 
regulatory analysis before they decide on a course of action. An agency needs to: 
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1. identify precisely what the problem is and present evidence of that problem; 

2. determine whether the market or another level of government could solve the 
problem; 

3. identify a broad range of approaches for solving the problem; 

4. examine the benefits and costs of each of those approaches; and 

5. determine if any of the options can solve the problem in a cost -beneficial 
manner. 

Of course, there is no way to guarantee that regulatory agencies will select the right 
solution eveiy time. But that doesn’t mean that agencies cannot do a much better job of 
regulating than they are doing now. As the Mercatus Regulatory Report Card project 
shows, agencies today are not doing a very good job of trying to figure out what will 
happen as a result of their regulations. There are ineffective regulations, regulations with 
unintended consequences, and regulations that impose excessive costs for less than 
satisfactory results on the books. As a result, the United States is experiencing 
overregulation. 

Today, the Code of Federal Regulations exceeds 163,000 pages. Agencies have failed to 
examine most of these rules by asking questions like: Which regulations are working and 
still necessary? Which ones are important and which ones are not? Which are directed at 
a problem that never materialized? Standards requiring retrospective review by agencies 
simply haven’t worked. 

This creates a terrible situation. The persistence of ineffective regulations absorbs so 
much time and resources that few can focus on those rules that truly are necessary and 
effective for protecting the public health, the environment, and the financial system. 

Even with the best of intentions, it is increasingly difficult for existing or new businesses 
to know if they are in full compliance with all the regulations applicable to them. 

In general, there is not much of an incentive for agencies to expend resources to identify 
and remove ineffective regulations from the Code of Federal Regulations. Tf agencies do 
not have the incentives or institutions to identify and remove those regulations that do not 
work, at the very least they can ensure, to the extent possible, that any new regulations 
are likely to accomplish their goals at reasonable costs. 

b. Wouldn’t Professor Glicksman’s approach require more, wasteful and redundant 
government spending on regulatory activity? 

The cost to agencies to regulate, although not insignificant, is a tiny fraction of what 
regulated entities and consumers must spend to comply with ill-considered rules. 
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c. Wouldn’t Professor Glicksman’s approach also lead to unnecessary and wasteful 
compliance costs for regulated entities? 

A very large portion of compliance costs for regulated entities are one-time expenditures, 
usually what are referred to as “fixed” and “sunk” costs. “Fixed” means that the costs do 
not vary with output. “Sunk” costs are one-time costs that firms do not get back later if 
the regulation changes. Once a regulation is enacted, firms are obligated to comply. 

Thus, once firms have made those compliance investments, taldng those requirements off 
of the books later does not help. Under an approach that errs on the side of regulation 
only to find it to be unnecessary, those resources have been spent and wasted. Such an 
approach, therefore, siphons limited resources away from efficient and effective 
investments in regulatory compliance and essential business operations. 

2. Have you ever felt pressured in any way to reach a particular conclusion in your 

research? How does the Mercatus Center endeavor to protect your academic freedom? 

When I worked for government, I absolutely felt pressured to reach specific conclusions. 

1 was often told to ensure that benefits exceed costs — not by changing decisions, but by 
changing my analysis. 

Unfortunately, my e.xperience was not unique, hi interviews with senior federal 
economists for a paper, 1 found that this type of pressure (to provide analysis showing 
benefits exceed costs) was a frequent occurrence. Sometimes the pressure was subtle, 
affecting one’s possibility for promotions. Other times it was overt. 

I have never felt pressured in any way to reach a specific conclusion at the Mercatus 
Center. Mercatus adheres to a Policy Regarding Independence of Research, as explained 
above. 


Both; 


Questions for Richard Williams and Jerry Ellig 

1 . How many members of Mercatus’ Board of Directors and officers have a relationship in any 
respect with Koch Industries or any of its related companies or foundations? 

Information regarding the membership of the Mercatus Center’s Board of Directors is 
available on our Web site, http://mercatus.org/all-people/1285. 
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2. According to an August 30, 2010 article in The New Yorker about the political activities of 
David and Charles Koch, the Koch Family Foundations founded and continue to run the 
Mercatus Center. 

What is your response to that statement? 

George Mason University Professor of Economics Tyler Covren is the general director and 
chairman of the board of directors for the Mercatus Center at George Mason University. The 
research and the testimony we provided are our own. We refer you again to the Mercatus 
Center’s Policy Regarding Independence of Research above. 

3. The New Yorker article quotes a Democratic political strategist describing the Mercatus 
Center as “ground zero for deregulation policy in Washington” and an environmental lawyer 
is also quoted in the article as saying the Mercatus Center is “a means of laundering 
economic aims.” 

What are your responses to these observations? 

Those opinions are misinformed and hence erroneous. The Mercatus Center seeks to ensure 
that sound and thorough economic analysis informs federal regulatory decision making. We 
assess the quality of agencies’ analyses via the Regulatory Report Card in order to highlight 
best practices and identify gaps. We also seek to supply missing information and analysis via 
research publications as well as Public Interest Comments submitted in regulatory 
proceedings by Mercatus scholars with relevant expertise. 

We engage in research driven analysis to question and test the role that different institutions 
play in creating prosperity. There are times when our findings encourage the role of 
government and times when they indicate that other institutions are more effective. 

4. What portion of Mercatus’ funding is derived from the Koch Family Foundations and other 
Koch-fmanced entities? 

We do research, not fundraising. Therefore, we do not know details of any giving to 
Mercatus. 

5. You admit that Mercatus’ score card is subjective or certainly gives the appearance of being 
so. Wouldn’t it be more useful to have an objective, impartial score card prepared by a 
totally independent entity? 

The Mercatus Center at George Mason University is an entirely independent entity that 
adheres to the highest academic standards. As explained above, the Regulatory Report Card 
is “subjective” according to standards for academically rigorous qualitative research. A 
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trained group of evaluators assess the quality and use of regulatory analysis according to a 
shared understanding of what kind of analysis deserves a 5, a 4, or any other score. 

The Regulatory Report Card was designed to fill a gap between two different approaches 
employed in prior scholarly evaluations of agency regulatory impact analysis. 

Some previous scholarly literature on “regulatory scorecards,” pioneered by the Government 
Accountability Office and by outside scholars such as Robert Hahn, used an objective “check 
the box” evaluation methodology that identified whether an agency’s analysis did various 
things enumerated in executive orders and 0MB guidance.'' This approach is informative and 
allows the researchers to compare large numbers of regulations. But it has a significant 
limitation: Checking off whether an agency did something does not tell us anything about the 
quality of what the agency did. The Office of Management and Budget explicitly noted this 
drawback in its discussion of regulatory scorecards in its 2008 report to Congress on the 
benefits and costs of federal regulations. We designed the Regulatory Report Card to 
respond to 0MB ’s concern. 

Much more intensive (and “subjective”) evaluations of the quality of individual regulatory 
analyses can be found in case studies that assess several individual regulatory analyses in 
depth." This approach has produced some excellent studies that evaluate quality in depth. The 
disadvantage of the case study method is that it is very resource-intensive, making it 
extremely difficult to compare the analyses of all economically significant regulations 
proposed in a year or over multiple years. 

The Mercatus Regulatory Report Card offers a more in-depth evaluation of quality than a 
purely objective check-the-box approach, but less depth than a case study. The advantage of 
this “middle ground” approach is that we can offer a qualitative evaluation of a large set of 
regulations. This allows us to highlight best practices and identify gaps in analysis that could 
benefit from further research. 

Any scholar or organization that wants to evaluate agency regulatory analysis faces this 
fundamental choice between breadth and depth. We think all of the approaches — the 
objective “check-the-box” approach, the case study approach, and our middle ground 
approach — have produced useful knowledge about the quality and use of regulatory analysis 

Wc cite a list of these studies in the report tliat was attached to Dr. Ellig's written testimony. Sec Jcrr\' Ellig and 
JolmMorrall. "Assessing ihe Qiialily of Regulalorv Analvsis; A New Evalualionand Data Set for Policy Research" 
Mercatus Center Working Paper (December 2010), p. 2. 

'■ Office of Management and Budget, 2008 Report to Congress on the Benefits and Costs of Federal Regulations and 
Unfunded Mandates on States, Local, and Tribal Entities. 

The most rccctit is Witiston Harringtoti, Lisa Hcitizcrlitig, and Richard D. Morgctistcni, Reforming Regulalorv 
Impact Analysis, Resources for the Future (April 2009). 
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in federal agencies. For that reason, we disagree that any one methodology should have a 
privileged position in the discourse over regulation. 

The question also asks if an “independent entity” should conduct an assessment like ours. 

The Mercatus Center at George Mason University is an academic research center unaffiliated 
with any federal agency, so we have no reason to favor one agency over another. 

We realize the question may be motivated by concern that the evaluators’ policy preferences 
may influence their evaluations of a regulation. When training the scoring team, we 
emphasize that all scorers should leave their own policy preferences at the door. The 
Regulatory Report Card is an assessment of how well federal agencies conduct and use 
regulatory analysis, not an assessment of whether any Mercatus researcher agrees with the 
agency’s decisions about the regulation or the underlying law. The paper submitted as part of 
Dr. Ellig’s testimony explicitly notes this: 

“Criteria 1-8 only evaluate the quality of regulatory analysis. We do not evaluate whether the 
proposed rule if economically efficient, fair, or otherwise good public policy. 

The same caveat applies to the Use criteria. Criteria 9 and 10 assess the extent to which 
analysis of the regulation’ s outcomes or benefits, the systemic problem, the alternatives, and 
costs informed the agency’s decisions about the regulation. On these criteria, we took great 
pains to avoid imposing the value judgment economists often make: that the agency should 
choose the most economically efficient alternative, as determined by a comparison of 
quantified benefits and costs. If an agency used some analysis of a regulation’s benefits to 
make decisions, even if it did not consider costs or efficiency, it could receive some points on 
Criterion 9. Similarly, if an agency demonstrated that it was fully cognizant of the net 
benefits of alternatives, but explicitly rejected the alternative with the greatest net benefits in 
favor of some other alternative for clearly articulated reasons, it could receive points on 
Criterion 10.”’ 

6. Can you e.xplain what the difference is in agency performance that you’ve graded a “0” 
versus a “ 1 ”? What about the difference between a “ 1” and a “2”? A “2” and a “3”? 

Page 7 of the paper submitted as part of Dr. Ellig’s written testimony includes a table that 
explains what each numerical score means. The table is reproduced below: 


Ellig and Morrall, supra note 4, at 8. 
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■ 

tWiiiplWf analysis of all or almost all aspects, with one or more “best 


Rrasoaiablv tlioraueh arulysis of iiiiist .iH'Xts aoil ur siiuws ai leatl uue ''best 
praclKe" 

3 

Reasonably thorough analysis of some aspects 

2 

Some relevant discussion with some docuiiiriitalinn of analysis 

1 

Perfunctory siatenii-nl w ilh little rs|ilanatiiin or tlncumvittalinii 

It 

l.ildc or no relevant cuiilenl 


The paper submitted as pan of Dr Ellig's testimony also notes that a more in-depth 
explanation of the scoring methodology is contained in a paper available on the Meicalus 
Center Web site that presents the 2008 results This paper includes examples of regulations 
tliat achieved a high score on the 'outcomes" criterion <EPA National Ambient Air Quality 
Standards for Lead), a low score (Dept of Veterans Affairs Post 9/1 1 Gl Bill), and a 
middling score (Labor Dept Cranes and Derricks) * 

We also have videos that we use for training scorers In these videos. Dr. Ellig explains each 
criterion and presents examples The videos are available at 
htti) //mcicatus ore/content/reitulntorv-renorl-card-video-iraininu . 


7 You gave the Fish and Wildlife Service (FWS) low .scores with regard to its 2009 to 2010 
Migratory Game Bird Hunting Regulations on several items that seem pretty arbitrary For 
example, it gets a 1/5 on "Does the analysis adequately assess uncertainty about the 
outcomes'’" What does that mean in this context? If FWS sets the hunting season, people 
will go out and buy shotgun shells and orange vests and shoot birds, and if it doesn’t, then 
people won't Please clarify what you meant by this assessment 

The relevant uncertainty is not w hether hunting will occur if the regulation is issued, but how 
much hunting will occur and how that will impact the bird population Tliese uncertainties 
affect the amount of consumer surplus the regulation can be expected to produce each year 


" Elllgaitd McLaughlin, sufttv note 1. at 7-12 
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and the sustainability of bird populations over time. The regulatory analysis, however, does 
not take these uncertainties into account. This omission is especially glaring given that (1) 
the agency re-uses the same regulatory impact analysis over several years and only updates it 
when new National Hunting and Fishing Survey data become available, (2) the regulation 
only provides “outside limits” for season lengths, bag limits, and locations; states establish 
the actual limits, and (3) the regulatory impact analysis considers only duck hunting, not all 
of the birds covered by the rule. Given these sources of uncertainty, some qualitative 
assessment of the range of possible outcomes and factors that influence them should have 
been possible. Since the department has taken a similar approach to this regulation and to the 
analysis for a number of years, some quantitative analysis of the accuracy of past predictions 
should have been possible. The analysis did receive 1 point because it acknowledged 
uncertainty about the amount of hunters’ willingness to pay for hunting and used a range of 
values for that variable. 

■ FWS gets a 0/5 on "Does the analysis adequately assess uncertainty about the 
existence or size of the problem?" You said there was "No relevant content." 
However, in earlier sections of the your report card, you gave FWS’s analysis a high 
score for having established the need for the rule (market failure) and that there have 
been studies of bird populations for more than 50 years. So how was FWS supposed 
to show with certainty why the rule is needed, AND to show uncertainty about the 
need for the rule? 

The Regulatory Report Card contains three questions about uncertainty related to 
outcomes, the systemic problem, and costs. To receive points on any of the Report 
Card questions about uncertainty, the regulatory impact analysis must explicitly 
consider possible uncertainties. The analysis does not have to identify big 
uncertainties in order to receive credit for analyzing uncertainty. If, after explicit 
consideration, the analysis concludes that the uncertainties are small or unimportant, 
and it presents evidence that backs up this claim, it can receive credit for analyzing 
uncertainty. If the analysis does not explicitly consider relevant uncertainties, then it 
does not receive credit for evaluating them. 

The regulatory impact analysis for the bird hunting regulation acknowledged no 
uncertainty about the existence or the size of the problem. Therefore, we could give 
no credit on the uncertainty question because the analysis did not appear even to have 
considered uncertainty about the existence or size of the problem. If the regulatory 
impact analysis had explicitly explained what uncertainties about the existence or size 
of the problem were considered and then presented evidence that they were small or 
unimportant, it would have received a better score on this criterion. Based on the 
information presented in the analysis, we cannot tell if the FWS analysts assessed 
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possible uncertainties about the existence or size of the problem and concluded they 
were small or if they neglected to consider these uncertainties at all. 

A critic might suggest that we should give agencies the benefit of the doubt and 
assume uncertainty is irrelevant if the agency did not explicitly analyze uncertainty. 
However, this approach would generate the absurd result of giving the same score to 
an analysis that ignored the uncertainty issue and an analysis that made a solid 
attempt at analyzing uncertainty. That approach would certainly not be fair to the 
agencies that actually do analyze possible uncertainties. 

■ FWS gets a 0/5 on “Does the analysis identify how the regulation would likely affect 
the prices of goods and services?” Mercatus said the analysis “estimates hunters’ 
expenditures on goods and services, but not effects of the regulation on prices.” So 
FWS is supposed to know how much Winchester is going to charge for shotgun shells 
this year? And is the price of shotgun shells (or orange vests, or decoys) really 
affected by a rule in which FWS sets hunting seasons? 

Considering that no legal bird hunting can occur unless FWS issues this rule, the rule 
probably does have a significant effect on the demand for and therefore the price of 
shotgun shells, vests, and decoys. This is, however, a very unusual regulation, and 
these kinds of price effects may be less important for evaluating this regulation than 
for evaluating most other economically significant regulations. 

Typically, regulations require or prohibit certain types of behavior. The requirements 
or prohibitions generate costs for the regulated entity, and some or all of these costs 
get passed on to consumers in the form of price increases. The price increases induce 
consumers to cut back consumption of the good or service whose price has increased. 
The increase in price, plus the value that consumers sacrifice when they use less of 
the good or service, is a cost associated with the regulation. Thus, the change in prices 
and the consumer response are important effects of typical regulations. They affect 
the total cost, and they also affect how costs are divided up between regulated entities 
and consumers. 

The bird hunting regulation is very atypical in this regard. It neither prohibits nor 
compels hunting; it merely allows the hunting season to occur. In a sense, this 
regulation reduces the “price” of hunting from very high (the risk of being caught 
hunting illegally if hunting is not permitted) to very reasonable (the price of a hunting 
permit). This is likely the biggest price change associated with the regulation, but it 


11 



116 


affects benefits rather than costs. The quality of the benefits analysis is addressed 
under criterion 5, outcomes. 

A critic might argue that we should refrain from scoring a regulatory analysis on its 
analysis of price effects when the agency issues a “permissive” regulation that does 
not contain price-increasing prohibitions or mandates. The broader issue is whether 
we should refrain from scoring an analysis on any criterion or sub-question that 
seems less relevant to the particular regulation. We considered this generic issue 
numerous times when evaluating individual regulations, whenever a particular 
criterion or question seemed less relevant to the particular regulation under review. 

We concluded that omitting some criteria for particular regulations would make the 
evaluation process less transparent and less informative. 

To ensure that all Regulatory Report Card evaluations are comparable, every 
regulation is evaluated on every criterion. We do not omit criteria or pass on 
evaluating a regulation on a criterion based on ad hoc judgments about how important 
we may think that criterion is for evaluating that particular regulation. The Report 
Card provides a description of what the regulatory analyses do and what they don’t 
do. 

We publicly furnish scores on each individual criterion (and each sub-question scored 
separately under the Analysis criteria). Readers who feel a particular criterion is not 
relevant, or not relevant for some regulations, are free to omit that criterion and 
recalculate the scores to see if the omission changes the results materially. 

8. In concluding that the Department of Transportation failed to adequately identify the problem 
it sought to address in promulgating the Motorcoach Crash Protection rule that would have 
required lap and shoulder seatbelts for passenger seats on new motorcoaches, you noted 
among other things that there was no demand for seatbelts among motorcoach users. Should 
“demand” for seatbelts on motorcoaches drive safety regulations? 

This question misstates what the Regulatory Report Card evaluation actually says on this 
topic. The Report Card evaluation of this regulation does not claim that there is no demand 
for seat belts among motorcoach users. The actual statement is, “The analysis does not show 
that motorcoach users are demanding seatbelts on motorcoaches and not being provided with 
them. The analysis even mentions that some motorcoach companies have started providing 
seatbelts in their motorcoaches.”® This statement merely reports what is in the regulatory 
impact analysis. The regulatory impact analysis does not show there is an unmet demand, and 


^ htt |)://!nc rc atus.orp/rcp ort caids/inotorcQac> 'i-ci ~a5h-protcct ion. 
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in fact, it suggests some evidence that at least some motorcoach users want seatbelts and 
manufacturers are supplying them. 

In asking whether consumer demand for seatbelts “should” drive safety regulations, the 
question steps well beyond the scope of the Regulatory Report Card. The Mercatus 
Regulatory Report Card is an assessment of how well federal agencies conduct and use 
regulatory analysis, not an assessment of whether any Mercatus researcher agrees with the 
agency’s decision-making criteria or decisions about the regulation. Section 1(b)(1) of 
Executive Order 12866 requires agencies to “identify the problem that it intends to address 
(including, where applicable, the failures of private markets or public institutions that warrant 
new agency action) as well as assess the significance of that problem.” In reporting that the 
regulatory impact analysis did not consider whether consumers “are demanding seatbelts on 
motorcoaches and not being provided with them,” we merely report that the regulatory 
impact analysis did not analyze whether there is a market failure. 

Decision makers ought to know whether consumers are getting the level of safety they are 
willing to pay for. Willingness-to-pay studies estimate how much consumers are willing to 
pay to obtain the benefits offered by regulation. Information on willingness-to-pay need not 
determine the regulator’s decision, but it is important information every decision maker 
ought to consider. If customers are not receiving the level of safety they are willing to pay 
for, this would likely indicate a market failure. The regulatory agency ought to find out why 
not, so the regulation can address this problem. A regulatory agency that did a thorough job 
of tracing this problem back to its root cause, using a coherent theory consistent with human 
behavior and empirical evidence that suggests the theory is true, would receive a high score 
on criterion 5, identification of the systemic problem. 

That does not mean this is the only definition of the systemic problem or way of analyzing 
the systemic problem that would receive a high score. Congress or the agency may decide (as 
the question below implies) that all customers should receive an equal level of safety 
regardless of their ability to pay. In that case, a relevant analysis of the systemic problem 
might compare the level of safety or safety features available on motorcoaches that primarily 
serve high-income passengers (perhaps charters) with the level of safety or safety features 
available on motorcoaches that primarily serve low-income passengers. If low-income 
passengers travel on less safe buses, and the analysis presented evidence that this reduced 
safety stems from their lower ability to pay for such safety, then the analysis could score well 
on definition of the systemic problem. 

Congress or the agency may also decide that consumer preferences for safety are essentially 
irrelevant; all consumers should be forced to pay for safety even if few consumers want to. In 
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this case, analysis of the systemic problem would start by identifying and justifying the level 
of safety Congress or the regulatory agency believes is appropriate and then explain and 
provide evidence why this level of safety is not achieved. 

Any of these three approaches could have earned a high score on criterion 5 if accompanied 
by a coherent theory and empirical evidence. 

9. With respect to the Department of Transportation’s Motorcoach Crash Protection rule, one of 
the reasons you provided for giving the DOT a low score for identifying and demonstrating 
the existence of a market failure or other systemic problem is that DOT did not provide an 
explanation of '‘why passengers are not getting the amount of safety they have shown 
themselves willing to pay for.” Should an individual’s willingness to pay for safety be a 
consideration at all, given that such an inquiry assumes an ability to pay? 

For reasons explained above, a high quality regulatory analysis should consider whether 
individuals are receiving the level of safety they are willing to pay for. If they are not, then 
there may be a market failure that regulation can correct. As we explain above, systemic 
problems may take forms other than market failures, and so there are multiple ways the 
regulation could have achieved a higher score for analysis of the systemic problem even 
though it did not identify a market failure. 

10. Are you familiar with the Administrative Conference of the United States'!’ Do you think that 
entity would be able to prepare an independent analysis of the costs and benefits of 
regulations and of agencies’ cost-benefit analyses? 

Although we are familiar with ACUS, our understanding is that it is not currently staffed 
with any economists. For an independent analysis by any entity, it is necessary to have 
economists who are thoroughly familiar with benefit-cost analysis and, more generally, the 
laws, executive orders and guidance (particularly 0MB Circular A-4) that guide production 
of these documents. It should be noted that regulatory impact analyses are much more than 
just benefit-cost analysis. They also include, for example: 

a. A description of the problem, including evidence that the problem is significant, 
systemic, and the result of a failure that necessitates government intervention. It 
should also assess where this condition exists and whether another level of 
government might be better for solving the problem. 

b. A baseline analysis that shows whether normal market activity is likely to solve 
the problem within a reasonable length of time. A baseline analysis examines the 
likely changes in consumer and producer knowledge and behavior (and resulting 
risk changes) from the current period into the near future. Changes in consumer 
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and producer behavior from the baseline form the basis for estimation of benefits 
and costs. 

c. A risk assessment. In order to do a benefit analysis for a regulation intended to 
reduce risk, it is necessary to know the beginning risk and how much each option 
would reduce the risk. 

d. A regulatory flexibility analysis. 

e. An analysis of the distributional effects of the regulation. This is particularly 
important to demonstrate where special interests may be driving unwarranted 
regulation. 

1 1 . With respect to the precise criteria that Mercatus uses in making its Report Card evaluations, 
they appear to not match up to the requirements of Executive Order 12866 or 0MB Circular 
A-4 on Regulatory Analysis, as they purport to do. 

Why did Mercatus chose to depart from the criteria outlined in those documents? 

In what respects did Mercatus did depart from existing criteria? 

What additional criteria did Mercatus consider? 

Is it fair to score an agency based on criteria that it was not required to follow? 

All of the criteria in the Regulatory Report Card are based on requirements in the 

presidential executive orders on regulatory analysis and 0MB Circular A-4. 

In November 2010, 0MB issued a Regulatory Impact Analysis Checklist for agencies to 
follow. The paper submitted as part of Dr. Ellig’s written testimony contains a crosswalk 
chart that shows how the criteria in the Regulatory Report Card correspond to the 0MB 
checklist. Ten of the 12 criteria in the Regulatory Report Card have direct analogs in the 
OMB checklist, which is based on Executive Order 12866 and Circular A-4. We have 
reproduced this crosswalk table below. 

Crosswalk of 2010 OMB Regulatory Impact Analysis Checklist with Mercatus Regulatory 

Report Card evaluation criteria 


OMB Checklist 

Mercatus Evaluation Criteria 

Does the RIA include a reasonably detailed 
description of the need for the regulatory 
action? 

Criterion 6: How well does the analysis 
demonstrate the existence of a market failure or 
other systemic problem the regulation is 
supposed to solve? 

Does the RIA include an explanation of how the 
regulatory action will meet that need? 

Criterion 5: How well does the analysis identify 
the desired outcomes and demonstrate that the 
regulation will achieve them? 
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OMB Checklist 

Mercatus Evaluation Criteria 

Does the RIA use an appropriate baseline (i.e., 
best assessment of how the world would look in 
the absence of the proposed action)? 

Criterion 7, question D: Does the analysis 
adequately assess the baseline (what the state of 
the world is likely to be in the absence of 
further federal action)? 

Is the information in the RIA based on the best 
reasonably obtainable scientific, technical, and 
economic information and is it presented in an 
accurate, clear, complete, and unbiased manner? 

Criterion 2: How verifiable are the data used in 
the analysis 

Criterion 3 : How verifiable are the models or 
assumptions used in the analysis? 

Criterion 4: Was the analysis comprehensible to 
an informed layperson? 

Criferion 3 includes an assessment of whether 
' the models and assumptions are based on peer- 
reviewed or otherwise reliable publications. 
However, the Mercatus evaluation does not 
assess the quality of the underlying science. 

Are the data, sources, and methods used in the 
RIA provided to the public on the Internet so 
that a qualified person can reproduce the 
analysis? 

Criterion 1 takes the first step by assessing how 
easily the RIA itself can be found on the 

Internet. 

Criteria 3 and 4 include an assessment of how 
easily the reader could find the underlying data, 
sources, and methods from information or links 
provided in the RIA or the Federal Register 
notice. 

To the extent feasible, does the RIA quantify 
and monetize the anticipated benefits from the 
regulatory action? 

Criterion 5, question 2: How well does the 
analysis identify how the outcomes are to be 
measured? 

To the extent feasible, does the RIA quantify 
and monetize the anticipated costs? 

Multiple questions under Criterion 8 (Benefits 
and Costs) assess how well the analysis 
identifies, quantifies, and monetizes costs. 

Does the RIA explain and support a reasoned 
determination that the benefits of the intended 
regulation justify its costs (recognizing that 
some benefits and costs are difficult to 
quantify)? 

Criterion 8, question F: Does the analysis 
identify the approach that maximizes net 
benefits? 

Criterion 8, question G: Does the analysis 
identify the cost-effectiveness of each 
alternative considered? 

Does the RIA assess the potentially effective 
and reasonably feasible alternatives? 

Criterion 7: How well does the analysis assess 
the effectiveness of alternative approaches? 
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OMB Checklist 

Mercatus Evaluation Criteria 

Does the preferred option have the highest net 
benefits (including potential economic, public 
health and safety, and other advantages; 
distributive impacts; and equity), unless a 
statute requires a different approach? 

Criterion 10: Did the agency maximize net 
benefits or explain why it chose another option? 

Does the RIA include an explanation of why the 
planned regulatory action is preferable to the 
identified potential alternatives? 

Criterion 9: Does the proposed rule or RIA 
present evidence that the agency used the 
Regulatory Impact Analysis? 

Criterion 10: Did the agency maximize net 
benefits or explain why it chose another option? 

Does the RIA use appropriate discount rates for 
the benefits and costs that are expected to occur 
in the future? 

Considered under Criterion 5, question 2 ( How 
well does the analysis identify how the 
outcomes are to be measured?), as well as in 
several questions about measurement and 
comparison of benefits and costs under 

Criterion 8 (Benefits and Costs). 

Does the RTA include, if and where relevant, an 
appropriate uncertainty analysis? 

Criterion 5, question E: Does the analysis 
adequately assess uncertainty about the 
outcomes? 

Criterion 6, question D; Does the analysis 
adequately assess uncertainty about the 
existence and size of the problem? 

Criterion 8, question E; Does the analysis 
; adequately address uncertainty about costs? 

Does the RIA include, if and where relevant, a 
separate description of the distributive impacts 
and equity (including transfer payments and 
effects on disadvantages or vulnerable 
populations)? 

Criterion 8, question H: Does the analysis 
identify all parties who would bear costs and 
assess the incidence of costs? 

Criterion 8, question 1: Does the analysis 
identify all parties who would receive benefits 
and assess the incidence of benefits? 

Does the analysis include a clear, plain- 
language executive summary, including an 
accounting statement that summarizes the 
benefit and cost estimates for the regulatory 
action under consideration, including the 
qualitative and non-monetized benefits and 
costs? 

Criterion 4: Was the analysis comprehensible to 
an informed layperson? 
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OMB Checklist 

Mercatus Evaluation Criteria 

Does the analysis include a clear and 
transparent table presenting (to the extent 
feasible) anticipated benefits and costs i 

(qualitative and quantitative)? 

Criterion 4: Was the analysis comprehensible to 
an informed layperson? 

Goals and measures to assess results of the 
regulation in the future - No content. i 

Criterion 1 1 : Does the proposed rule establish 
measures and goals that can be used to track the 
regulation’s results in the future? 

Provisions for gathering data to assess results 
of the regidation in the fiiiure - No content. 

Criterion 12: Did the agency indicate what data 
it will use to assess the regulation’s 
performance in the future and establish 
provisions for doing so? 


Each of the four Analysis criteria (Criteria 5-8) has a series of sub-questions that are 
separately scored and then averaged to produce the total score for that criterion. Each of 
these sub-questions is derived from Executive Order 12866 or Circular A-4. This is 
documented in a published pilot study that used these criteria to evaluate the quality of 
homeland security regulations finalized in the years 2003 to 2007.^^ We reproduce these 
questions and documentation below: 

Criterion 5: Outcomes 

• Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life?” 

• Does the analysis identify how these outcomes are to be measured?^^ 

• Does the analysis provide a coherent and testable theory showing how the regulation will 
produce the desired outcomes?^^ 

• Does the analysis present credible empirical support for the theory? 

• Does the analysis adequately assess uncertainty about the outcomes?” 

Criterion 6: Systemic Problem 


Jaiiiic Bclcorc and Jcn\’ Ellig, “Homeland Security and Regulatory Analysis; Are We Safe Yet?/' Riiigers Law 
Jowmrv/ 40:1 (Fall 2008).' 

“Tn constructing measures of "effectiveness," final outcomes, such as lives saved or life-vears saved, are preferred 
to measures of intennediate outputs, such as tons of pollution reduced, crashes avoided, or cases of disease 
avoided. ' Circular A-4 at 12. 

“Even when a benefil or cosl caiuiot be expressed in monelarv imils, you should sliU try to measure il in terms of 
its physical units. If it is not possible to incasnrc the physical units, yon should still desenbe the benefit or cost 
qualitatively." Circular A-4 at 9. 

“Each agency sliall base its decisions on the best reasonably obtainable scienlific. lecluiicaL economic, and other 
iiifonnatioii conceming the need for, and consequences of, the intended regulation or guidance docunicnt." Exec. 
Order 12866, Sec. 1.7. “Explain how llie actions required by the rule are linked to the expected benefits. For 
example, indicate how additional safety equipment will reduce safety hsks." Circular A-4 at 2. 

'“‘Exec. Order 12866, Sec. 1.7. 

“I Y|our analysis should include two fimdameiital components: a quantitative analysis cliaraclerizing tlie 
probabilities of the relevant outcomes and an assignment of economic value to the projected outcomes." Circular A- 
4 at 40. 
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• Does the analysis identify a market failure or other systemic problem?^^ 

• Does the analysis outline a coherent and testable theory that explains why the problem 
(associated with the outcome above) is systemic rather than anecdotal? 

• Does the analysis present credible empirical support for the theory? 

• Does the analysis adequately address uncertainty about the existence or size of the 
problem?^''’ 

Criterion 7: Alternatives 

• Does the analysis enumerate other alternatives to address the problem?^'^ 

• Is the range of options considered narrow (e.g., some exemptions to a regulation) or 
broad (e g., performance-based regulation vs. command and control, market mechanisms, 
nonbinding guidance, information disclosure, addressing any government failures that 
caused the original problem)'^^^ 

• Does the analysis evaluate how alternative approaches would affect the amount of the 
outcome achieved?^^ 

• Does the analysis adequately address the baseline, that is, what the state of the world is 
likely to be in the absence of federal intervention not just now but in the future?^^ 


“(1) Each agency sliall identify in vv riling the specific market failure (such as externalities, market power, lack of 
information) or other specific problem that it intends to address (including, where applicable, the failures of pubhc 
institutions) that warrant new agency actioiL as well as assess the significance of lliat problem, to enable assessment 
of whether any new regulation is wananted. 

(2) Each agency shall examine whether existing regulations (or other law) have created, or eontributed to, the 
problem lliat a new reguLition is intended to correct and whether those regulations (or other Law) should be modified 
to achieve the intended goal of regulation more effectively.” Exec. Order 1 2866 Sec. 1 . 

' "If the regulation is designed to correct a significant market failure, you should describe the failure both 
qualitatively and (where feasible) quantitatiy ely . . . For other inlerventions, you should also provide a 
demonstration of compelling social purpose and the likelihood of effective action. Although intangible rationales do 
not need to be quantified, the analysis should present and evaluate the strengths and limitations of the relevant 
arguments for these intangible values.” Circular A- at 4. 

' ^ ‘The precise consequences (benefits and costs) of regulatory' options are not always known for certain, but tlie 
probability of their occurrence can often be developed. The important uncertainties connected with vour regulalorv 
decisions need to be analyzed and presented as part of the overall regulatory' analysis.” Circular A-4 at 38. 

" ' (3) Eaeh agency shall identify' and assess available alternatives to direct regulation, including providing economic 
incentives to encourage the desired behavior, such as user fees or marketable permits, or providing information upon 
wliich choices can be made by the public Exec Order 12866 Sec. I "Even wlicre a market failure clearly exists, 
you should consider other means of dealing w ith the failure before turning to Federal regulation. Alternatives to 
Federal regulation include antitrust enforcement, consumer -initialed litigation in the product liabihtv system, or 
administrative compensation sv'stcnis.” Circular A-4 at 6. 

!! 

Agency regulatoiy' analysis is to include "An assessmenl. including the imderlying analy sis, of benefits anticipated 
from the rcgulatotv action (such as, but not limited to, the promotion of the efficient functioning of the economy and 
private markets, the enliancemenl of health and safely, the protection of the natural environment, and the elimination 
or reduction of discrimination or bias) together with, to the extent feasible, a quantification of those benefits;” Exec. 
Order 12866 Sec. 6(a)(3)(C)(i). “A regulation may be appropriate when you liave a clearly identified measure tliat 
can make govermnent operate more efficiently . In addition. Congress establishes some regulatory programs to 
redistribute resources to select groups. Such regulations slxmld be examined to ensure that they arc both effective 
and cost-cffcctivc.” Circular A-4 at 5. 
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Criterion 8: Costs and Benefits 

• Does the analysis identify and quantify incremental costs of all alternatives considered?^"* 

• Does the analysis identify all expenditures likely to arise as a result of the regulation?^^ 

• Does the analysis identify how the regulation would likely affect the prices of goods and 
services?^ 

• Does the analysis examine costs that stem from changes in human behavior as consumers 
and producers respond to the regulation?” 

• If costs are uncertain, does the analysis present a range of estimates and/or perform a 
sensitivity analysis?^* 

• Does the analysis identify the alternative that maximizes net benefits?” 

• Does the analysis identify the cost-effectiveness of each alternative considered?^” 

• Does the analysis identify all parties who would bear costs and assess the incidence of 
costs?”' 


”■ 'This baseline should be the best asscsstnciit of the wav the world w-ould look absent the proposed action. ” 
Circular A-4 at 15. 

'When you liave idenlilied a rai^e oralleraatives (e.g., dtCferenl levels of siringenev). vou should detennine ihe 
cost-cffcctivcncss of each option compared with the baseline as well as its incremental cost-cffcctivcncss compared 
with successively more stringent requirements.” Circular A-4 at 11. 

” Agency rcgulatoiy analysis is to include "An assessment, including the tindcrlving anaivsis, of costs anticipated 
from the regulatory action (such as, but not hmited to, the direct cost both to the govermnent in administering the 
regulation and to businesses and others in complying with the regulation, and any adverse effects on the efficient 
functioning of the economy, private markets (including productivitv, employment, and competitiveness), health, 
safety, and the natural enviromnent), together with, to the extent feasible, a quantification of those costs;” Exec. 
Order 12866, Sec. 6(a)(3)(C)(u). 

Id. OMB Circular A-4 adds, "You should include these effects in vour anaivsis and provide estimates of their 
monetary v'alucs w'hcn they arc significant: 

• Privale-sector conqiliance cosls and savings; 

• Goveniment administrative costs and savings; 

• Gains or losses in consumers' or producers' suipluses; 

• Discomfort or inconvenience costs and benefits: and 

• Gains or losses of time in w ork, leisure and/or coimnutiiig/travcl settings.” Circulai A-4 at 37. 

hi. 

'‘It is usually necessarv' to provide a sensitivitv analysis to reveal whether, and to what extent, the results of the 
analysis are sensitive to plausible cliangcs in the main assumptions and numeric inputs.” Circulai A-4 at 3 . Rules 
withaminal benefits or costs exceeding $1 billion require a quantitative analysis of uncertaintv. Circular A-4 at 41- 
42. 

"Each agency shall assess both the costs and the benefits of the intended regulation and, recognizing that some 
costs and benefits are difficult to quantify, propose or adopt a regulation onl}' upon a reasoned determination that the 
benefits of the intended regulation justify its cosls.” E.xec. Order 12866 Sec. 1.6. 

‘When an agenev determines that a regulation is the best available method of achieving the rcgulatoiy' objective, it 
sliall design its regulations in the most cost-effective mamier to achieve the regulatoiy objectri'e.” Exec. Order 
12866 Sec. 1.5. “Bothbenefil-cosl anaivsis (BCA) and cosl-elTecliveness anaivsis (CEA) provide a svslematic 
framework for identifying and evaluating the likely outcomes of alternative rcgulatoiv choices. A major nilcmakiiig 
should be supported by both Ivpes of analysis wherever possible. Specifically, you should prepare a CEA for aU 
major nilcmakings for wdiicli the primary' benefits arc improved public health and safety to the extent tliat a valid 
effectiveness measure can be developed to represent expected health and safety' outcomes .” Circular A-4 at 9. 

“Your regidatory analysis should provide a separate description of distributional effects (i.e.. how bothbeneilts 
and costs arc distributed among sub-populations of particular conccni) so that decision makers can properly consider 
them along with the effects on economic efficiency.” Circular A-4 at 14. 
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• Does the analysis identify all parties who would receive benefits and assess the incidence 
of benefits?^^ 

The only Regulatory Report Card criteria not mentioned in the 0MB checklist are two 
criteria that assess whether the agency provided for retrospective analysis of the regulation’s 
actual effects after adoption; Criterion 1 1 (measures and goals) and Criterion 12 
(retrospective data). Although ex post, retrospective analysis has not received as much 
attention as ex ante analysis of proposed regulations. Section 5 of Executive Order 12866 
states that agencies should conduct retrospective analysis. Section 6 of President Obama’s 
executive order on regulatory review requires agencies to develop plans for periodic 
retrospective review of regulations. The Government Performance and Results Act arguably 
requires retrospective analysis of regulations,^^ and the GPRA Modernization Act of 2010 
appears to require retrospective analysis of regulations that contribute toward high-priority 
agency or government goals. The GPPA. Modernization Act requires identification of every 
program, tax expenditure, and regulation that contributes toward high priority goals, and 
progress toward these goals must be reviewed quarterly. 

The regulatory impact analysis is an agency’s best attempt at projecting the expected 
outcomes and costs of a proposed regulation. It seems logical that the regulatory impact 
analysis is also the right place to lay the groundwork for retrospective analysis. Since the 
regulatory impact analysis predicts benefits and costs, it provides a basis for setting goals 
and establishing measures. Since the regulatory impact analysis is supposed to be based on 
empirical analysis, it is the logical place to identify data that could be used for retrospective 
analysis. 

Some agencies do this. For example, in the 2008 regulatory impact analysis on collection of 
biometric data under the US-VISIT program, the Department of Homeland Security 
explained how the regulation advances departmental strategic goals and indicated the 
measures that would be used to track the performance of the regulation. This regulation 
received a score of 5 on both retrospective analysis criteria. 

The Regulatory Report Card gives partial credit on the retrospective analysis criteria when 
the outcomes or costs projected in the regulatory impact analysis could readily be used to 
establish goals for or measure the regulation’s results, even if the agency did not commit to 
doing so. It also gives partial credit if the agency committed to reexamining the regulation’s 


Id. 

Jen^- Brito and Jem’ Ellig, "Toward a More Perfect Union: Regulator}- Analysis and Performance Management, ' 
S Florida Slate l'niversify> Business Review 1 (2009). 

Department of Homeland SecuriW, "Air-Sea Biometric Exit Project regulatory Itnpact Analysis” (April 17, 2008), 
Appendix D and E. 
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results sometime in the future, even if the agency did not explicitly identify the goals, 
measures, or data it will use. 

Finally, readers who are still not convinced that the two retrospective analysis criteria are 
useful or appropriate are free to recalculate the scoring results without them. We publish 
each regulation’s score on each criterion separately, so that readers who do not think a 
particular criterion is relevant can see if omitting that criterion substantially changes the 
scoring results. 

12. Professor Glicksman observes that mechanisms like variances, exceptions, and waivers 
already exist that allow stakeholders to seek adjustments to rules after they have been 
promulgated on a case-by-case basis, allowing agencies to respond to concrete problems 
raised by a particular rule in a particular context, rather than being forced to speculate about 
myriad potential problems with rules that have not yet been promulgated. 

What is your response? 

This position is based on three presumptions. One, in all cases the basic regulatory approach 
chosen by the agency is the correct one and that only minor adjustments would sufficiently 
address any problem that may arise. Two, agencies have the capacity to respond to, in some 
cases, hundreds of thousands of firms across the United States after promulgation of a rule. 
Three, small- and medium-size firms affected by the rule would ask for a variance. 

These considerable presumptions are simply not warranted. For instance, during the course of 
Dr. William’s career he had opportunities to work with small- and medium-sized firms. In 
many cases, they showed a marked reluctance to engage government authorities. Thus, these 
mechanisms should not be relied upon as substitutes for high quality analysis that informs 
decision makers regarding the likely effects of a regulatory proposal. 

13. Professor Glicksman notes that cost-benefit analysis is indeterminate, maldng costs and 
benefits difficult to predict on the front-end of the rulemaking process. 

What is your response? 

First, characterizing benefit-cost analysis as indeterminate is only a problem if one believes 
that this or any other kind of analysis will be used as the sole basis for making dispositive 
decisions. We do not agree that the results of a benefit-cost analysis (or, more correctly, a 
regulatory impact analysis) should replace an informed decision making process. We do 
believe agencies should analyze before they decide. 

More generally, this characterization of benefit-cost analysis as being uncertain can be 
applied to all science, not just economics. Science always contains uncertainty; that is the 
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nature of science. In fact, virtually all inputs into the regulatory process have some degree of 
uncertainty. 

For example, risk assessments based on animal models may contain estimates of risk that 
vary by many orders of magnitude (e.g., the risk may vary from 1/100,000 to 1/100,000,000). 
This does not mean that risk assessments are not useful in helping to reach a decision. 
Uncertainty in science can be reduced by more research. In fact, for regulations that are 
likely to have large impacts on the economy, additional research to reduce uncertainty is 
often the right choice as it may help to make a more informed decision. 

As another example, regulatory agencies are concerned with whether or not their 
interpretation of statutes will survive judicial review. Agency attorneys advise decision 
makers about the likelihood of an agency prevailing if its regulatory strategy is subject to 
legal challenge. As with scientific input, this legal advice contains some degree of 
uncertainty, but that uncertainty does not eviscerate its value. 

Whether related to markets or ecologies, prediction is a difficult exercise. Benefit-cost 
analysis is somewhat easier than more macro or global models because it limits the scope of 
the predictive model to small changes in relatively few markets. The fact that it is difficult 
does not negate its usefulness. If we do not have information on what the likely impacts of 
our rules are, we are just regulating in the dark, hoping for a good outcome but as likely to 
fail as to succeed. 

Benefit-cost analysis serves decision makers by asking and answering questions that many in 
the regulatory process fail to ask. For example, to do a benefit-cost analysis, economists must 
answer questions such as: How is this solution (mechanism) expected to solve part or all of a 
problem? How much of the problem will it solve and how long will it take? How much will 
consumers be willing to pay for this kind of solution? What other kinds of risk will emerge as 
a result of this action and how large will they be? What will the cost be, both now and in the 
future? How will it affect domestic competition and international competitiveness? Is there a 
better way to do this? What if we don’t do anything, will the problem solve itself? Who is 
likely to benefit from this regulation; who will lose? Do the benefits outweigh the costs? Are 
there parts of the regulations we could change and not reduce much of the benefits but a lot 
of the costs? These are valid questions that help us to be more certain the action we are 
taking will help to solve social problems. 

Good economists, like good risk assessors, will identify their assumptions and assess how 
changing them will change the answers. They will identify how much uncertainty is 
contained in each key parameter. In many cases, it is possible to quantify uncertainty, and 
this should be done for significant regulations. These types of analyses (uncertainty) can be 
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invaluable for telling decision makers when it is worthwhile to obtain more information 
before making a decision. 

14. The OOP's budget proposal includes slashing $99 million from the Occupational and Safety 
Health Administration, a 40 percent reduction in the budget of the federal agency most 
responsible for making sure the nation's workplaces are safe - Democrats claim that 
translates into 8,000 fewer workplace hazard inspections and 740 fewer whistleblower 
discrimination probes. 

What is your response? 

We believe budgeting decisions should be made based on outcome-oriented performance 
measures, rather than just good intentions or measures of activity like inspections or 
enforcement actions. Congress first needs to understand whether past regulatory expenditures 
have actually improved workplace safety. This should be proven with data, not just accepted 
on faith. That is one reason why sound retrospective analysis of regulation is important, and 
presumably why Congress, in the GPRA Modernization Act, directed agencies to identify 
regulations that contribute to achievement of high priority goals. 

If past expenditures have not been effective, legislators cannot presume that additional 
expenditures will be effective. If past expenditures have been effective, legislators need to 
know if additional expenditures can be expected to produce additional benefits. Once 
Congress obtains this information, it is then in a position to consider what effect budget cuts 
or increases are likely to have on workplace deaths or injuries, and lawmakers can weigh 
these expected results against other spending priorities. 

1 5. There are various regulations that impose standards for coal dust in mines. As you know, 
coal dust, which is highly flammable, may have been a contributing cause of the Massey coal 
mine explosion in West Virginia, which took the lives of 29 miners. In fact, next month will 
mark the one-year anniversary of that explosion. 

Would better regulatory analysis have saved those miners’ lives? 

We have not studied the specific facts of the Massey tragedy, so our answer must necessarily 
be general . 

Sound, thorough regulatory analysis would help identify whether this is a systemic, 
nationwide problem in all coal mines, a problem unique to mines in West Virginia, or an 
isolated case. If it is not a nationwide, systemic problem, regulatory analysis could help 
improve safety by focusing agency attention on the types of coal mines where the problem is 
most likely to e.xist. 
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One of the regulations reviewed as part of the Mercatus Regulatory Report Card project is a 
proposed 2010 regulation intended to lower miners’ exposure to respirable coal mine dust.^’ 
The regulatory impact analysis did not consider the potential effect of coal mine dust on 
explosions. If this effect turned out to be a significant “co-benefif’ from reducing respirable 
coal mine dust, it might have justified a different approach to the regulation. 

1 6. Do you think better regulatory analysis could have prevented the Triangle Shirtwaist Fire that 
occurred in New York City 100 years ago this month that resulted in the deaths of 146 
individuals? 

We have not studied the specific facts of the Triangle Shirtwaist Fire, so our answer must 
necessarily be general. 

Regulatory analysis Is a process for organizing and evaluating information about the pros and 
cons of alternative policy tools. Sound regulatory analysis helps in evaluating the 
effectiveness and cost-effectiveness of the alternatives identified to address to public policy 
objectives, such as improved public and workplace safety. 

17. Should agencies be compelled to wait until a tragedy like the Triangle Shirtwaist Fire occurs 
before they can act? 

Regulatory analysis does not require inaction. Risk can be estimated either by examining 
systemic epidemiologic evidence or underlying structural conditions for large, rare events. 
The latter would be done for nuclear power plants, for example, and would also make sense 
for assessing the risk factors for catastrophic fires. These types of evidence are also used in 
regulatory analysis. Agencies can do risk assessments and benefit-cost analysis using the 
tools we have discussed even without epidemiological data (e g., accidents, poisonings) to 
see if there is a potential problem; whether markets are addressing the problem; which level 
of government should handle the problem (if markets are not addressing it); what other risks 
might arise from addressing the problem; and which options appear to be the best to prevent 
the problem from manifesting. 

18. Do you think better regulatory analysis could have prevented the BP oil spill? 

We have not studied the specific facts of the BP oil spill, so our answer must necessarily be 
general. As stated in response to earlier questions, we believe that sound, thorough analysis 
can be extremely valuable in helping to solve problems effectively and efficiently. 


imp :.//m crcat us.or g/rc pQrtcaids/j o \vcriiie-m incr s -c xposiuc- r cspirabic-cortl-miiK‘ -dus t. 
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19. Please provide a list of every agency scoring a “0” or a “1” in your assessment of whether the 
agency properly Identified a problem needing a regulatory solution. 

The Regulatory Report Card assesses the quality and use of analysis for individual 
regulations. We have submitted complete sets of scoring data for 2008 and 2009 for the 
record. Below, we attach a list of regulations that received a score of 0 or 1 in 2008 or 2009 
on criterion 6, identification of the systemic problem. Also listed is the department that 
issued each regulation. We include the scores on the four sub-questions (A-D). Scores on the 
four sub-questions are averaged and then rounded to produce the overall score for the 
criterion. 


26 



131 


Proposed_Rule_08 

RIN_08 

Dept_08 

Q6_08 

Q6A_08 

Q6B_08 

Q6C_08 

Q6D_08 

Nondiscrimination in State/Locai 

Government Services 

1190-AA46 

DOJ 

0 

0 

1 

0 

0 

Nondiscrimination by Pubiic/Commercial 
Faciiities 

1 1 90-AA44 

DOJ 

0 

0 

1 

0 

0 

Hospital Inpatient Prospective Payment 
Systems 

0938-AP15 

HHS 

0 

1 

0 

0 

0 

Abandoned Mine Land Program 

1029-AC56 

Interior 

0 

0 

0 

1 

0 

Federai Perkins Loan Program 

1840-AC94 

ED 

0 

1 

0 

0 

0 

Prospective Payment System for Long- 

0983- 

HHS 

0 

■\ 

0 

0 

0 

Term Care Hospitais 

A094 






Medicaid Program Premiums and Cost 
Sharing 

0938- 

A047 

HHS 

0 

0 

0 

0 

0 

Prospective Payment System for Skilled 
Nursing Facilities 

0938-AP11 

HHS 

0 

0 

0 

0 

0 

Post-9/11 Gl Bill 

2900-AN10 

VA 

0 

1 

0 

0 

0 

Time and Place for a Hearing before an 

0960- 

SSA 






Administrative Law Judge 

AG61 






HIPAA Code Sets 

0958-AN25 

HHS 

1 

3 

1 

1 

0 

Refuge Alternatives for Underground 
Coal Mines 

1219-AB58 

Labor 

1 

1 

1 

1 

0 

Energy Conservation Standards for 
Fluorescent Lamps 

1904-AA92 

Energy 

1 

2 

1 

1 

0 

Changes to the Outpatient Prospective 
Payment System 

0938-AP17 

HHS 

1 

1 

1 

0 

0 

Oil Shale Management - General 

1004-AD90 

Interior 

1 

2 

1 

0 

0 

HIPAA Electronic Transaction Standards 

0938- 

AM50 

HHS 

1 

1 

1 

1 

0 

Employment Eligibility Verification 

9000-AK91 

FAR 

1 

1 

1 

0 

0 

Teacher Education Assistance Grant 
Program 

1840-AC93 

ED 

1 

1 

0 

1 

0 

Medicare Program: Revisions to 
Physician Fee Schedules 

0938-AP18 

HHS 

1 

1 

2 

2 

0 
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Proposed_Rule_08 

RIN_08 

Dept_08 Q6_08 

Q6A_08 

Q6B_08 

Q6C_08 

Q6D_08 

Stale Flexibility for Medicaid Benefit 
Packages 

0938- 

A048 

HHS 1 

2 

0 

0 

0 

Proposed Hospice Wage Index for Fiscal 
Year 2009 

0938-AP14 

HHS 1 

2 

2 

1 

0 

Schedule of Fees for Consular Services 

1400-AC41 

State 1 

1 

0 

1 

0 

CHAMPUS/TRICARE 

0720-AB22 

Defense 1 

1 

1 

2 

1 


Proposed_Rule_09 

RIN_09 

Dept_09 

Q6_09 

Q6A_09 

Q6B_09 

Q6C_09 

Q6D_09 

General and Non-Loan Programmatic 

1840-AC99 

ED 

0 

0 

0 

0 

0 

Issues 

Children's Health Insurance Program 

0938-AP53 

HHS 

0 

0 

0 

0 

0 

(CHIP) 

Housing Trust Fund Program 

2506-AC23 

HUD 

0 

1 

0 

0 

0 

Expansion of Enrollment in the VA 

2900-AN23 

VA 

0 

0 

0 

0 

0 

Health Care System 

State Fiscal Stabilization Fund Program 

1810-AB04 

ED 

0 

0 

0 

0 

0 

Positive Train Control 

2130-AC03 

DOT 

0 

0 

0 

1 

0 

Portland Cement NESHAP 

2060-A015 

EPA 

0 

1 

0 

0 

0 

Hospice Wage Index for FY 2010 

0938-AP45 

HHS 

1 

1 

0 

1 

0 

Loan Guarantees for Projects that 

1901-AB27 

DOE 

1 

2 

1 

1 

0 

Employ Innovative Technologies 

Weatherization Assistance Program 

1904-AB97 

DOE 

1 

3 

2 

0 

0 

Hospital Inpatient and Long-Term Care 

0938-AP39 

HHS 

1 

1 

1 

0 

0 

Prospective Payment System 

Outpatient Prospective Payment 

0938-AP41 

HHS 

1 

1 

1 

0 

0 

Revisions to Payment Policies Under the 

0938-AP40 

HHS 

1 

1 

0 

1 

0 

Physician Fee Schedule 

Revisions to the Medicare Advantage 

0938-AP77 

HHS 

1 

2 

1 

2 

0 


Program 
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Proposed_Rule_09 RIN_09 Dept_09 Q6_09 Q6A_09 Q6B_09 Q6C_09 Q6D_09 

Credit Assistance for Surface 2105-AD70 DOT 1110 0 

Transportation Projects 

Investing in Innovation 1855-AA06 ED 1 1 1 0 0 

Renewable Fuels Program 2060-A081 EPA 10 0 12 

Race to the Top Fund 1810-AB07 ED 1 1 1 0 0 

Electronic Health Record Incentive 0938-AP78 HHS 1110 0 

Program 

School Improvement Grants 1810-AB06 ED 1 2 1 2 0 

Motor Vehicle Safety Standards, Ejection 2127-AK23 DOT 10 0 11 

Mitigation 


29 
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Response to Post-Hearing Questions from Robert L. Glicksman, J.B. and Maurice 
C. Shapiro Professor of Environmental Law, George Washington University Law 
School 


Questions for the Record 
Ren. Howard Coble 
Chairman 

Subcommittee on Courts. Commercial and Administrative Law 
For the Hearing on “Raising the Agencies’ Grades - Protecting the Economy. Assuring 
Regulatory' Quality and Improving Assessments of Regulatory Need” 

March 29. 2011 


1. How much of your testimony is informed by your own, real-world 
experience gained first-hand in a government agency? 

I have acquired extensive experience studying statutes and the issues that arise when agencies 
implement them in my 30 years as a law professor specializing in environmental and 
administrative law. The following is a partial list of my publications: 

Books 

Administrative Law: Agency Action in Legal Context (Foundation Press 2010) (with R. 
Levy). 

Public Natural Resources L.aw (4 volumes, ThomsonAVest, updated three times annually) 
(2d ed. 2007) (with G. Coggins) (1st edition published in 1991. 

Risk Regulation At Risk: Restoring a Pragmatic Approach (Stanford University Press 
2003) (with S. Shapiro). 

Environmental Protection: Law and Policy (6iiied. Aspen Publishers 2011) (with D. 
Markell, B. Buzbee, D, Mandelkcr, & D. Bodansky) ( (previous editions published in 1999, 

2003, 2007). 

Pollution Limits and Polluters' Efforts to Comply: The Role of Government 
Monitoring and Enforcement (Stanford University Press, forthcoming 2011) (with D. 
Earnhart). 

Modern Public Land Law in a Nutshell (3d ed. West Group) (2006) (yy'ith G. Coggins) 
(previous editions published in 1995, 2001). 

Regulation of Toxic Substancf.s and Hazardous Waste: Cases, Materials, Notes, and 
Que.stions (1992) (unbound) (awarded Postlethwaite Research Prize, May 1992). 

A Proposed Strategy to Prevent Groundwater Contamination in Kansas (University of 
Kansas 1986) (with G. Coggins) (report prepared under contract to the Kansas Department of 
Health and Environment). 


Book Chapters 
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The Role of Executive Power in Environmental Law: The Tak e Care Clause and the Unitary 
Rxccutive Theory , in CONTEMPORARY PRINCIPLES IN CONSTITUTIONAL Environmental Law 
(James May ed., American Bar Ass’n & Environmental Law Inst.) (forthcoming). 

Facing Unprecedented Stewardship Challenges: Climate Change and Federa l Land Management, 
in Global Warming: A Reader (W, Rodgers et ah, cds.) (forthcoming). 

The Failure of II-S. Climate Chance Policy (with C. Schroeder) and Anatomy of Industry 
Resistance to Climate Change: Running the Script , in ECONOMIC Thought AND U.S. 

Climate Change Policy (The MIT Press, D. Driesen, ed. 2010). 

Federal Preemption bv Inaction , in PREEMPTION CHOICE: TPtE THEORY , Law AND Reality of 
Federalism's Core Question (W. Buzbee ed., 2009). 

.Securing Judicia l R eview of Agency Inaction (and Action) in the Wake of Norton v. Southern 
Utah Wilderness Alliance in Strategies for Environmental Success in an 
Uncertain Judicial Climate (M. Wolf, ed.) (2005). 

A New Progressive Agenda for Public Health and the Environment ©. Schroeder & R. 
Stcinzor eds. 2005) (listed as one of ten “primary authors”). 

NEPA Litigation and The Environmental Review Process , in D.MaNDELKER, NEPA Law ANT) 
Litigation (revised chapters for 2004-2007 releases) 

Scone of the Environmental Impact Statement , in D.Mandelker, NEPA Law and Litigation 
(revised chapters for 2006-2007 releases). 

The CEC’s Biodiversity Conservation Agenda in GREENING NAFTA: THE North American 
Commission for Environmental Cooperation (D. Marked & I. Knox, eds.) (2003). 

Municipal Regulation of Hazardous Waste in A. Rathkoff, The Law of ZONING AND 
Pl.anning chapter 7A (Release # 72, 6/2000) (with A. Dan Tarlock). 

Wildlife and Habitat Protection in ENVIRONMENTAL Law Practice Guide: State and Federal 
Law chapter 24 (M. Gerrard ed., 2d ed. 1998). 

The Impact of Federal Environmental Protection Laws on Water Allocation in the 26th Annual 
Institute on Planning, Zoning, & Eminent Domain chapter 7 (1996), reprinted in 34 Public 
Land & Resources Law Digest 309 (1997). 

Public Resources in environment.al Law and Practice: Compliance/Litigation/Forms 
chapter 14 (D. Stever & E. Dolin eds., 1995) (with G. Coggins). 

Federal Groundwater Quality Control Law and Policy in proceedings of the Natural Resources 
Law Center conference on Water Quality Control; "Integrating Beneficial Use and 
Environmental Protection” (1988). 
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Municipal RcaulaLion of Hazardous Waste in A. Ratiikopf, The Law of ZONING AND 
Planning, chapter 7A (4th ed. 1987) (with A. Dan Tarlock). 

Interstate Compacts for Low-Level Radioactive Waste Disposal: A Mechanism for Excluding 
Out-of-State Waste in LOW-LEVEL RADIOACTIVE WASTE REGULATON - SCIENCE, POLITICS, AND 

Fear chapter 3 (M. Burns ed., 1987). 

Law Review Publications 

Agency-Specific Precedents, 89 TEXAS L. Rev. 499 (201 1) (with R. Levy), 

Climate Change Adaptation: 4 Collective Action Perspective on Federalism Considerations, 40 
Envtl. L. 1159(2010). 

The Constitution, the environment, and the Prospect of Enhanced Executive Power, 40 Envtl. 
L.REP. 1102(2010), 

Science. Politics, Law and the Arc of the Clean Water Act: The Role of Assumptions in the 
Adoption of a Pollution Control Landmark, 32 WASH. U. J.L. & POL’ Y 99 (2010) (with M. 
Batzcl). 

to Courts and Preemption of State Remedies in Collective Action Perspective, 59 C.ASE. 
W. Res. V. L. Rev. 919 (2009) (with R, Levy), 

Ecosystem Resilience to Disruptions Linked to Global Climate Change: An Adaptive Approach 
to Federal Land Management, 87 Neb, L, Rev, 833 (2009), 

Sustainable Federal Land Management: Protecting Ecological Integrity and Preserving 
Environmental Principal, AATCISAL.}. 147 (2008), 

A Collective Action Perspective on Ceiling Preemption by Federal Environmental Regulation: 
The Case of Global Climate Change, 102 N,W, U, L. Rev. 579 (2008) (with R. Levy). 

Coal-Fired Power Plants, Greenhouse Gases, and State Statutory Substantial Endangerment 
Provisions: Climate Change Comes to Kansas, 56 U. KAN. L. Rev. 517 (2008). 

Effectiveness of Government Interventions at Inducing Better Environmental Performance: Does 
Effectiveness Depend on Facility of Firm Characteristics?, 35 B.C. Envtl, Aff. L. Rev. 479 
(2008) (with D, Eamhart), 

Bridging Data Gaps through Modeling and Evaluation of Surrogates: Use of the Best Available 
Science to Protect Biological Diversity Under the National Forest Management Act, 83 IND. L.J. 
465 (2008). 

Balancing Mandate and Discretion in the Institutional Design of Federal Climate Change 
Policy, 102 Nw. U. L. REV. COLLOQUY 196 (2008). 
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Nothinfi h Real: Protecting the Regulatory Void through Federal Preemption by Inaction, 26 
Va. Envtl. LJ. 5 (2008). 

The Comparative Effectiveness of Government Interventions on Environmental Performance in 
the Chemical Industry, 26 STANFORD Envtl. L.J. 317 (2007) (with D. Eamhart). 

Depiction of the Regulator-Regulated Entity Relationship in the Chemical Industry: Deterrence- 
Based V. Cooperative Enforcement. 31 Wm. &Mary Envtl. L. & Pol’yRev. 603 (2007) (with 
D. Earnhart). 

Global Climate Change and the Risks to Coastal Areas from Hurricanes and Rising Sea Levels: 
The Costs of Doing Nothing, 52 LOYOLA L. REV. 1 1 27 (2006). 

From Cooperative to Inoperative Federalism: The Perverse Mutation of Environmental Law and 
Policy, 41 Wake Forest L. Rev. 719 (2006). 


Justice Rehnquist and the Dismantling of Environmental Law, 36 ENVTL. L. REP. 10585 (2006) 
(with J. May). 

Improving Regulation Through Incremental Adjustment, 52 Kan. L. Rev. 1 179 (2004) (with S. 
Shapiro). 

Traveling in Opposite Directions: Roadless Area Management Under the Clinton and Bush 
Administrations, 34 Envtl. L. 1 143 (2004). 

The APA and the Back-End of Regulation: Procedures for Informal Adjudication, 56 AdMTN. L. 
Rev. 1 159 (2004) (with S. Shapiro), 

The Value of Agency-Forcing Citizen Suits to Enforce Nondiscretionary Duties, 10 WlDENER L. 
Rev. 353 (2004). 

Chevron, State Farm, and the EPA in the Courts of Appeals During the 1990s, 3 1 ENVTL. L. 
Rep. 10371 (2001) (with C. Schroeder), reprinted in 32 LAND USE AND Env’t L. Rev. 327 
( 2002 ). 

Making A Nuisance of Takings Law, 3 WASH. U. J. L. & Pol’Y 149 (2000). 

Goals, Instruments, and Environmental Policy Choice, 10 DukeEnvtl. L. & POL’Y F. 297 
(2000) (with S. Shapiro). 

Federal Environmental Law in the "New" Federalism Era, 30ENVTL. L. Rep. 11122 (2000) 
(with S. McAllister). 

State Liahiliiv for Environmental Violations: The U.S. Supreme Court’s "New" Federalism, 29 
envtl. L. Rep. 10665 (1999) (with S. McAllister). 
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Wilderness in Context, 76 DfiN\'. U. L, Rev. 383 (1999) (with G. Coggins), 

Hardrock Minerals, Energy Minerals, and Other Resources on the Public Lands: The Evolution 
of Federal Natural Resources Law, 33 TULSA L.J. 765 (1998) (with G. Coggins). 

Concessions Imw and Policy in the National Park System, 74 Deny. U. L. Rev. 729 (1997) (with 
G. Coggins). 

Fear and Loathing on the Federal Lands, 45 U. Kan. L. Rev. 647 (1997). 

Regulatory Reform and (Breach of) the Contract With America: Improving Environmental 
Policy or Destroying Environmental Protection?, 5 K.AN. J. L. «& PUB. POL Y # 2 (Winter 1996), 
at 9 (with S. Chapman). 

Pollution on the Federal Lands IV: Liability for Hazardous Waste Disposal, 12 UCLA J. E.NVTL. 
L. & Pol'y233 (1994). 

Pollution on the Federal Lands 111: Regulation of Solid and Hazardous Waste Management, 13 
St,\nford Ekvtl. L.J. 3 (1994). 

Pollution on the Federal Lands II: Water Pollution Law, 1 2 UCLA J . Envtl. L. & POL'Y 6 1 
(1993). 

Pollution on the Federal Lands I : Air Pollution Law, 12 UCLA J. Envtl. L. & POL'Y 1 (1993). 

Watching the River Flow: The Prospects for Improved Interstate Water Pollution Control, 43 
W.ASH. U. J. Urb. & CONTEMP. L. 1 19 (1993) (awarded Postlethwaite Research Prize, May 
1 994). 

EPA and the Courts: Twenty Years of Law and Politics, 54 J. OF LAW & CONTEMP. PROBS. 249 
(1991) (with C. Schrocder), reprinted in 24 Land USE & Env't L. Rev. 289 (1993) (awarded 
Postlethwaite Research Prize, May 1992). 


Judicial Activism and Restraint in the Supreme Court's Environmental Law Decisions, 42 
Vanderbilt L. Rev. 343 (1989) (with R. Levy) (chosen in W. Rodger.s, Environmental Law 
48 (2d ed. 1994) as one of the 25 best law' review articles on environmental law ever written). 

To the Promised Land: A Cenlury> of Wandering and A Final Homeland for the Due Process and 
'Faking Clauses, 68 OREGON L. REV. 393 (1989) (with M, Davis), reprinted in 22 L.ANDUSE& 
Env't L. Rev. 211 (1991) (awarded the Rice Prize as the best University of Kansas law faculty 
article in 1990). 


Congress, the Supreme Court, and the Quiet Revolution in Administrative Law, 1988 Duke L.J. 
819 (w'ith S. Shapiro). 
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A Retreat from Judicial Activism: The Seventh Circuit and the Environment, 63 Chi.-Kent L. 

Rev. 209 (1987). 

Groundwater Pollution It: An Immodest Proposal for a Strategy to Prevent Future Groundwater 
Pollution, 35 U. Kan. L. Rev. 241 (1987) (with G. Coggins). 

Groundwater Pollution I: The Problem and the Law, 35 U. Kan. L. Rev. 75 (1986) (with G. 
Coggins). 

Federal Preemption and Private Legal Remedies for Pollution, 134 U. PA. L. Rev. 125 (1985) 
(awarded the Rice Prize as the best University of Kansas law faculty article in 1986). 

,4 Guide to Kansas Common Law Actions Against Industrial Pollution Sources, 33 U. Kan. L. 
Rev. 62 1 (1985) (awarded the Rice Prize as the best University of Kansas law faculty article in 
1985). 

Allocating the Cost of Constructing Excess Capacity: “Who Will Have to Pay for It All?", 33 U. 
Kan. L. Rev. 429 ( 1 985), reprinted in VllI Public Utilities Law Anthology (1984- 85). 

Severability and the Realignment of the Balance of Power over the Public Lands: The Federal 
Land Policy and Management Act of 1976 After the Legislative Veto Decisions, 36 HASTINGS 
L.J. 1 (1984). 

Federal Recreational Land Policy: The Rise and Decline of the Land and Water Conservation 
Fund, 9 COLUM. J. Envtl. L. 125 (1984) (with G. Coggins). 

2. Please respond to Mr. Ellig’s statement that regulatory impact analysis is 
“nothing more than Decision Making 101 applied to regulation.” 

Strict cost-benefit analysis is not Decision Making 101. Strict cost-benefit analysis involves the 
quantification and monetization of all costs and benefits regardless of whether the costs and 
benefits can suitably be quantified and then reduced to monetary terms (e.g., lives saved or 
endangered species protected). Then, the analyst must compare various regulatory alternatives to 
determine the one that ma.ximizes net benefits (a fairly complicated economics concept). 

Conveniently, proponents of cost-benefit analysis strategically alternate between which “kind” of 
cost-benefit analysis they are advancing. In some contexts, they describe cost-benefit analysis as 
a kind of harmless, rough, apples-to-oranges comparison of benefits and costs. 1 agree that this 
would be more akin to Decision Making 101. But, when it comes to attacking agency decision- 
making, suddenly the cost-benefit analysis they are using is the strict cost-benefit analysis 
described above — quantification, monetization, and identification of the alternative that 
maximizes net benefits. As 1 explained above, this is nothing like Decision Making 101, and is 
instead an approach to analysis that is employed by a small field of highly trained professionals. 
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Even a cursory inspection of the Mercatus Center’s Regulatory Report Card project reveals that 
the project is about ensuring that regulatory agencies engage in strict cost-benefit analysis, rather 
than the kind of rough, apples-to-oranges balancing connoted by Decision Making 101, 

For a fuller discussion of these issues, see: 

Frank Ackerman & Lisa Heinzerling, Priceless: On Knowing the Price of Everything 
AND THE Value of Nothing (2004). 

Sidney A, Shapiro & Robert Glicksman, Risk Regulation At Risk: Restoring a 
Pragmatic Approach (2003). 

3, Please respond to Mr. Ellig’s suggestion that Congress should require all 
agencies to explain “how major elements of regulatory analysis affected 
decisions about regulation.” 

Federal agencies arc already required to provide such explanations by myriad statutes and 
executive orders, and they do so in the preamble to every major federal regulation. Requiring 
still more explanation of this sort would not only be duplicative and wasteful, but would delay 
agency regulatory decisionmaking without improving the quality of agency regulatory decisions. 

4. Please respond to Mr. Ellig’s contention that agencies essentially use 
regulatory analyses as post-hoc rationalizations for the decision to 
regulate, rather than as a basis for deciding whether to regulate. 

1 subscribe to a different view of the role that regulatory impact analyses play in regulatory 
decisionmaking. The factors that agencies are supposed to consider when making regulatory 
decisions are very clear: Congress spells out these factors in the relevant authorizing statute, 
pursuant to which an agency issues a regulation. These factors might include the costs of a 
proposed action, its feasibility, its faimess, and the benefits it will achieve for members of the 
public in both qualitative and quantitative terms. The agency’s analysis of these factors is 
demanding and involves careful consideration of the public comments it receives, which often 
number in the hundreds or thousands. To conduct this analysis, agencies employ an 
interdisciplinary team of professionals with expertise on each of the issues relevant to the 
decisionmaking — scientists, lawyers, and economists, for example. The explanations that 
accompany proposed rules routinely and that are derived from this process set forth various 
alternatives means of achieving the agency’s goals and solicit public comment on w'hich of these 
or other alternatives the agency has not yet considered are preferable. Agencies then choose 
from among the alternatives based in part on public comments and the experts reactions to them, 
llypotheiically speaking, if an agency’s views were to have solidified before the issuance of a 
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proposed rule, il is then the function of judicial review to remedy the situation. Specifically, the 
review'ing court can remand back to the agency any rule that docs not reflect a legitimate 
consideration of relevant statutory factors or that ignored relevant public feedback, and courts 
take this rule seriously. 

Unfortunately, when it comes time to prepare the regulatory impact analysis, the relevant 
statutory factors included in the statute and the interdisciplinary expertise of the agency s 
rulemaking team are often shoved aside, as economists with a myopic perspective prepare a 
distorted, purely economic analysis of the rule. 

For more information about how the regulatory impact analysis process interferes with the 
proper functioning of agency regulatory decisionmaking, see Rena Stemzor et al., A Return to 
Common Sense: Protecting Health, Safety, and the Environment Through "Pragmatic 
Regulatory Impact Analysis” (Ctr. for Progressive Reform, White Paper 909, 2009), available at 
hti|r:/,tw\vw.nrt>grcss i vereforin.t)rt’/articlcs/PRI.A 909. pdt . 

5. Mr. Ellig says that the goal of regulatory analysis is to “produce 

know'ledge about reality that can inform decisions.” You’ve said that 
regulatory analysis is indeterminate. What is your response to Mr. Ellig s 
assertion? 


As explained in my testimony, cost-benefit analysis necessarily requires predictions about how a 
particular regulation will affect a future, complex world. While some of these impacts can be 
anticipated, it is impossible to identify them with the specificity required for cost-benefit 
analysis. In the end. the form of analysis involves so much uncertainty, that it produces very 
little useful “knowledge about reality.” 

6. If mechanisms currently exist to help cure any problems with rules after 
they are promulgated, why do you think there has been such a push to add 
analytical requirements to the front-end of the rulemaking process? 

The push to add analytical requirements to the front-end of the rulemaking process that either 
duplicate pre-existing statutory requirements or force agencies to consider factors that are 
iiTelevant under the substantive statute arises largely not from any legitimate concern about 
improving the quality of regulation, but instead from efforts to delay the rulemaking process — a 
phenomenon referred to as “paralysis by analysis.” Often, regulated industries support efforts to 
delay the rulemaking process because they benefit from such delay. Every day that a regulation 
is delayed is literally money in the bank for regulated industry because the need to incur 
compliance costs is pushed into the future. 
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These addilional analytical requirements also provide regulated industries with additional 
avenues for weakening rules, thereby reducing compliance costs. These avenues are particularly 
attractive to regulated industries bccau.se they tend to be less transparent than back-end 
processes, better enabling industry to leverage its extensive resources to pursue regulatory relief. 

7. Mr. Williams says that regulatory agencies lack the kind of feedback 
mechanism that exists in the private sector that would help agencies 
distinguish between well-informed decisionmaking and “pre-determined’’ 
decisionmaking. What is your response? 

As I explained in my testimony, the traditional Administrative Procedure Act (APA) ailemaking 
process provides several effective feedback mechanisms to help guide and inform agency 
regulatory decisionmaking. These mechanisms include the notice-and-comment process and the 
judicial review process. Furthermore, 1 described how agencies can use feedback they receive 
during the rulemaking process to make incremental adjustments to regulations at the ’‘back-end” 
of the process. These adjustments can include the grant of exceptions, time extensions, 
variances, and waivers. Finally, agencies can use feedback they receive once a regulation has 
been implemented for several months or years to revise the rule through a subsequent rulemaking 
action. In some cases, statutes actually require agencies to periodically review and revise 
existing rules, as is the ease with the Clean Air Act’s National Ambient Air Quality Standards 
(NAAQS) program for criteria air pollutants. 

I also question whether the private sector’s feedback mechanisms arc an appropriate model for 
informing agency decisionmaking. For example, the National Commission on the Deepwater 
Florizon Oil Spill and Offshore Drilling final report found that BP had received several warning 
signs that it was in danger of losing control of its well. Nevertheless, BP failed to heed these 
warning signs, and as a result its action led to one of the costliest environmental disasters in U.S. 
history. 


8. Mr. Williams suggests that agencies should make their regulatory analyses 
publicly available for public comment. What is your respon.se? 

Agencies already make their regulatory analyses available for public comment, as part of the 
AP.A nolice-and-comment rulemaking process. For example, the Center tor Progressive Reform 
recently commented on the EPA’s cost-benefit analysis for its proposed coal ash disposal rule. 
The problem is that these regulatory analyses are hyperteehnical documents beyond the ken of 
the general public, 'v'ery few members have the required expertise to read and comment 
effectively on these documents, allowing regulated entities, who can afford to hire costly experts, 
to dominate the notice-and-comment process. 
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For discussion of the ways in w'hich access to agencies before, during, and after the promulgation 
of regulations is skewed heavily in favor of regulated entities, sec Wendy E. Wagner, Katherine 
Barnes, & Lisa Peters, Rulemaking in the Shade: An Empirical Study of EPA's Air Toxic 
Emission Standards, 63 ADMINISTRATIVE Law^ Review 99 (201 1 ). 

9. Please explain the costs of hamstringing an agency’s ability to promulgate 
health and safety regulations with needless analytical requirements. 

In my testimony, 1 provided an example of the costs that arose when the Occupational Safety and 
Health Administration's (OSHA) Cranes and Derricks Rule was delayed for several years while 
the agency struggled to complete all the unnecessary analytical requirements. By OSHA s ow'n 
estimates, every year the rule remained stuck, 53 people died and another 155 were injured 
unnecessarily. 


10. Why do you suppose two former EPA Administrators who served under 
Republican presidents felt compelled to publicly decry House 
Republicans’ attempts to gut the EPA? 

1 think the tw'O former Administrators w'ere appalled at the efforts being made by some House 
Republicans to turn back the clock on more than 40 years of environmental improvement 
achieved by the EPA under statutes such as the Clean Air and Clean Water Acts. I think they 
w anted to make it clear that protecting the health and safety of the American people should be a 
predominant, bipartisan concern of both parties (as it was throughout much of the 1970s), rather 
than having vocal elements of the Republican Party reflexively oppose any and all environmental 
protection initiatives and disguise their contempt for those initiatives by inaccurately demonixing 
them as “job-killing regulations.” In their Washington Post op-ed, the two former EPA 
Administrators said they were upset that Republicans in Congress were focusing only on the 
costs of the EPA's regulations without also considering the benefits of these regulations. 

Whether one conducts a strict cost-benefit analysis or a less quantified analysis, it is quite clear 
that the benefits of these regulations far outweigh their costs. In March 201 1 , for example, the 
EPA's Office of Radiation concluded that its centralized benefits estimate for Clean Air Act 
regulations issued by the agency under the 1990 amendments to the statute exceed costs by a 
factor of more than 30 to one, and that its high benefits estimate exceeds costs by 90 times. 

EnvtI. Protection Agency, Office of Radiation, Final Report, The Benefits and Costs of the Clean 
Air Act from 1990 to 2020 (March 2011). In other words, the EPA's regulations have been a 
W'onderful bargain for the United States. Nevertheless, Republicans were focusing entirely on 
costs in order to advance their agenda of gutting the EPA’s budget and legal authority to address 
pressing environmental issues such as global climate change. The two former EPA 
Administrators w'ere concerned that Republican’s misguided attacks on the EPA risked undoing 
many of the environmental successes the United States has achieved over the last 40 years. 
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1 1 . Last October, Rep. Fred Upton, who is now Chairman of the House 
Finergy and Commerce Committee, said that the EPA’s revised ozone 
standard would cost 7 million jobs and about $1 trillion annually and 
Senate Republicans warned that “nursing homes, schools and even 
doughnut chains could suffer under EPA climate regulations.” Is this 
typical of the kind of “costs” that would be assessed on the front-end of 
the rulemaking process, especially by those w'ho oppose regulation in 
principle? 

The numbers cited by Rep. Fred Upton sound like the kind of inflated or apocalyptic numbers 
that regulated industry and regulatory opponents often bandy about in order to halt or weaken 
health, safety, and environmental protection regulations. Invariably, when these numbers are 
analyzed in greater detail, they are found to be completely flawed. Inflated cost estimates have 
become more common in recent years, and I expect to sec even more of them in the future. The 
problem with these kinds of numbers is that they interfere with rational assessment of regulatory 
decisionmaking, rather than advance it, which I lake to be the point of cost-benefit analysis. 
Unfortunately, the Mercatus Center’s Regulatory Report Card project does nothing to address 
this problem. If anything, the project makes it worse by attempting to elevate the importance of 
the role that ex ante cost-benefit analyses play in regulatory decisionmaking. 


o 



